Jul 27 19:23:39.885693 ip-10-0-131-59 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 27 19:23:39.885802 ip-10-0-131-59 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 27 19:23:39.885818 ip-10-0-131-59 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 27 19:23:39.886260 ip-10-0-131-59 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 27 19:23:49.947228 ip-10-0-131-59 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 27 19:23:49.947242 ip-10-0-131-59 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5e8cc893a0f54602aa6c9264035934cc --
Jul 27 19:25:47.791841 ip-10-0-131-59 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 19:25:48.182687 ip-10-0-131-59 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:25:48.182687 ip-10-0-131-59 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 19:25:48.182687 ip-10-0-131-59 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:25:48.182687 ip-10-0-131-59 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 19:25:48.182687 ip-10-0-131-59 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:25:48.185155 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.185005    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 19:25:48.191681 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191661    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:25:48.191681 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191680    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191685    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191689    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191692    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191695    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191698    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191701    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191705    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191707    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191711    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191714    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191717    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191719    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191722    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191725    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191727    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191730    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191732    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191737    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191741    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:25:48.191753 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191744    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191747    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191750    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191752    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191755    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191759    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191762    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191765    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191768    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191772    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191775    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191778    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191781    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191784    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191786    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191789    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191791    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191794    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191796    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:25:48.192297 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191799    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191801    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191804    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191807    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191809    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191812    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191814    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191817    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191819    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191822    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191825    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191828    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191831    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191834    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191836    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191839    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191843    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191846    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191848    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191851    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:25:48.192766 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191854    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191857    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191859    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191862    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191864    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191867    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191870    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191873    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191875    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191878    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191880    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191883    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191886    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191888    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191891    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191893    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191896    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191900    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191904    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:25:48.193267 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191907    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191910    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191912    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191915    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191917    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191920    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191922    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191925    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191929    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191932    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191936    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191938    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191941    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191944    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191947    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191949    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191952    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191954    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191957    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.191960    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:25:48.193723 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192720    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192728    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192731    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192734    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192736    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192740    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192742    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192745    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192748    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192750    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192753    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192756    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192759    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192762    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192765    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192767    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192771    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192774    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192776    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192780    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:25:48.194188 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192783    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192786    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192788    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192791    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192795    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192798    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192800    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192803    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192805    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192808    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192810    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192813    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192815    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192818    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192822    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192825    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192828    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192831    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192836    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:25:48.194699 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192839    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192842    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192845    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192848    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192850    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192854    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192856    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192859    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192862    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192864    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192867    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192869    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192872    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192875    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192877    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192880    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192883    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192885    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192889    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192892    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:25:48.195156 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192895    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192898    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192900    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192903    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192905    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192907    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192910    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192912    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192915    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192917    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192919    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192922    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192925    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192928    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192930    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192933    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192935    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192938    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192941    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192943    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:25:48.195724 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192945    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192948    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192950    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192953    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192955    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192958    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192961    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192963    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192966    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192968    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192971    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192973    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192976    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192978    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192981    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192984    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192986    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192989    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192991    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:25:48.196586 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.192993    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193400    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193405    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193408    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193411    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193414    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193417    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193420    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193422    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193425    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193428    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193431    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193433    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193436    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193439    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193441    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193444    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193446    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193450    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193452    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:25:48.197111 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193455    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193458    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193460    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193464    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193468    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193471    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193474    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193478    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193480    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193483    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193486    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193489    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193492    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193495    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193498    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193501    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193504    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193506    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193509    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:25:48.197664 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193512    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193514    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193518    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193523    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193525    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193528    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193531    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193533    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193536    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193538    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193541    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193543    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193546    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193549    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193551    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193555    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193559    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193562    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193564    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:25:48.198196 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193567    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193569    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193572    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193575    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193577    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193581    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193583    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193586    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193588    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193591    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193593    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193596    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193599    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193602    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193605    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193607    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193609    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193612    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193614    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193617    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:25:48.198728 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193619    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193625    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193627    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193630    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193633    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193635    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193638    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193640    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193643    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193645    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193648    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193650    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193653    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193656    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193658    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193661    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193663    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193665    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193668    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193670    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:25:48.199258 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193673    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.193676    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193746    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193753    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193760    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193765    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193770    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193773    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193777    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193782    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193785    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193789    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193792    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193797    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193800    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193804    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193807    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193810    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193813    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193816    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193821    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193824    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193827    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193830    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 19:25:48.199830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193833    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193837    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193840    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193843    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193847    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193851    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193853    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193857    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193860    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193863    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193867    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193870    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193873    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193876    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193879    2579 flags.go:64] FLAG: --enable-server="true"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193882    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193887    2579 flags.go:64] FLAG: --event-burst="100"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193890    2579 flags.go:64] FLAG: --event-qps="50"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193893    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193896    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193899    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193905    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193908    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193911    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193914    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 27 19:25:48.200472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193917    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193920    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193923    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193926    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193929    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193932    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193935    2579 flags.go:64] FLAG: --feature-gates=""
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193939    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193942    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193945    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193948    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193951    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193954    2579 flags.go:64] FLAG: --help="false"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193957    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193960    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193963    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193965    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193969    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193973    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193975    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193978    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193981    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193984    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 19:25:48.201133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193987    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193990    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193993    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193996    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.193999    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194005    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194008    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194011    2579 flags.go:64] FLAG: --lock-file=""
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194013    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194016    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194019    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194024    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194027    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194030    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194033    2579 flags.go:64] FLAG: --logging-format="text"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194036    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194039    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194042    2579 flags.go:64] FLAG: --manifest-url=""
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194045    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194050    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194052    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194056    2579 flags.go:64] FLAG: --max-pods="110"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194060    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194062    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194066    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 19:25:48.201782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194069    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194072    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194075    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194078    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194086    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194089    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194092    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194095    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194098    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194104    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194107    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194110    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194114    2579 flags.go:64] FLAG: --port="10250"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194120    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194123    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06d7b2981d4314813"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194126    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194129    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194132    2579 flags.go:64] FLAG: --register-node="true"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194135    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194139    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194142    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194145    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194148    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194152    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194155    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 19:25:48.202461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194158    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194161    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194163    2579 flags.go:64] FLAG: --runonce="false"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194166    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194169    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194172    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194175    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194178    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194181    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194184    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194187    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194190    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194193    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194196    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194199    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194202    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194206    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194208    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194214    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194218    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194222    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194226    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194229    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194232    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194235    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 19:25:48.203239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194253    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194258    2579 flags.go:64] FLAG: --v="2"
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194264    2579 flags.go:64] FLAG: --version="false"
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194270    2579 flags.go:64] FLAG: --vmodule=""
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194274    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.194277    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194402    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194406    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194409    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194411    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194414    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194416    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194420    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194422    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194424    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194427    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194429    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194432    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194435    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194437    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194440    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:25:48.203948 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194443    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194445    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194447    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194451    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194453    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194457    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194461    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194464    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194466    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194469    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194471    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194474    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194476    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194479    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194481    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194484    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194486    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194489    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194492    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194494    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:25:48.204523 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194497    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194499    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194502    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194504    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194507    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194510    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194512    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194515    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194518    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194521    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194525    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194528    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194530    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194532    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194535    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194537    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194540    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194545    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194549    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:25:48.205053 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194552    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194554    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194557    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194560    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194562    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194565    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194567    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194570    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194572    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194575    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194577    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194581    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194585    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194588    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194591    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194593    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194596    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194599    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194602    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194604    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:25:48.205625 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194607    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194610    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194612    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194615    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194617    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194620    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194622    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194626    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194630    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194633    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194637    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194641    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194644    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194646    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194649    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194651    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194654    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194656    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194659    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194661    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:25:48.206153 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194664    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194666    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194669    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194671    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.194674    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.195268    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.202348    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.202364    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.202387    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.202396    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202456    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202461    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202466    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202469    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202472    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:25:48.206707 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202475    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202479    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202482    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202485    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202488    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202491    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202494    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202498    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202501    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202504    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202506    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202511    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202514    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202517    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202520    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202523    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202526    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202529    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202531    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202534    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:25:48.207123 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202537    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202542    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202546    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202549    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202553    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202556    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202560    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202562    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202565    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202568    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202571    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202573    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202576    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202579    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202581    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202584    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202587    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202590    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202593    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:25:48.207652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202596    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202599    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202602    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202605    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202608    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202612    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202616    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202620    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202622    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202625    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202628    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202631    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202634    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202636    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202639    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202641    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202644    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202648    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202651    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202653    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:25:48.208157 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202656    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202659    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202662    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202665    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202667    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202670    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202673    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202676    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202679    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202681    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202684    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202687    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202691    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202695    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202698    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202701    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202704    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202707    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202710    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202713    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:25:48.208694 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202715    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202718    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202721    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202723    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202726    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202728    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202731    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202734    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202736    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202739    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202742    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202745    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202748    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202751    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202754    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:25:48.209222 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.202760    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202888    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202893    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202896    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202901    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202905    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202908    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202912    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202915    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202918    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202921    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202924    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202928    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202931    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202933    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202938    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202941    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202944    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202947    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:25:48.209652 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202950    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202953    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202956    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202959    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202962    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202964    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202967    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202970    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202973    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202976    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202978    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202981    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202984    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202987    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202990    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202992    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202995    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.202997    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203000    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203003    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:25:48.210158 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203005    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203009    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203013    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203016    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203018    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203021    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203025    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203027    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203030    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203033    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203036    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203038    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203041    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203044    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203046    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203049    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203052    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203054    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203057    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203060    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:25:48.210834 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203063    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203066    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203068    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203071    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203074    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203076    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203079    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203082    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203084    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203087    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203090    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203092    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203095    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203098    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203100    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203103    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203106    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203108    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203111    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203114    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:25:48.211682 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203117    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203119    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203122    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203124    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203127    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203130    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203132    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203135    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203137    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203140    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203143    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203145    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203148    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203151    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203153    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203156    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203159    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203162    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203164    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203167    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:25:48.212538 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:48.203170    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:25:48.213207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.203175    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:25:48.213207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.203296    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 19:25:48.213207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.205446    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 19:25:48.213207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.206847    2579 server.go:1019] "Starting client certificate rotation"
Jul 27 19:25:48.213207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.206958    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 19:25:48.213207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.207727    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 19:25:48.213207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.209556    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 19:25:48.216460 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.216436    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 19:25:48.232188 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.232161    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 27 19:25:48.234385 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.234368    2579 log.go:25] "Validated CRI v1 image API"
Jul 27 19:25:48.235360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.235341    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 19:25:48.238810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.238790    2579 fs.go:135] Filesystem UUIDs: map[655f8c83-8f28-45ec-b4a7-1ca1ca1850a0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a71920f5-cab0-4d0b-afe8-10aef42d0842:/dev/nvme0n1p3]
Jul 27 19:25:48.238867 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.238810    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 19:25:48.244403 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.244271    2579 manager.go:216] Machine: {Timestamp:2026-07-27 19:25:48.242878733 +0000 UTC m=+0.334230225 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101799 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2079777cae645b799e258b6d55b6f4 SystemUUID:ec207977-7cae-645b-799e-258b6d55b6f4 BootID:5e8cc893-a0f5-4602-aa6c-9264035934cc Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:8d:4b:be:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:8d:4b:be:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:15:cd:cd:77:b1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 19:25:48.244403 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.244398    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 19:25:48.244519 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.244486    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 19:25:48.247343 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.247299    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 19:25:48.247502 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.247347    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-59.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 19:25:48.248172 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.248161    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 19:25:48.248205 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.248175    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 19:25:48.248205 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.248183    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 19:25:48.248810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.248798    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 19:25:48.249628 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.249608    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 19:25:48.249670 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.249619    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 19:25:48.251481 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.251469    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 19:25:48.251596 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.251586    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 19:25:48.251643 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.251617    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 19:25:48.251643 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.251628    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 19:25:48.251643 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.251643    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 19:25:48.251745 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.251655    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 27 19:25:48.251745 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.251664    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 19:25:48.252050 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.252036    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 19:25:48.252094 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.252061    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 19:25:48.255294 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.255275    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 19:25:48.257255 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.257240    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 19:25:48.257348 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.257272    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 19:25:48.258900 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258886    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258903    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258911    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258917    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258926    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258932    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258940    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258946    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258954    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 19:25:48.258969 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258961    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 19:25:48.259331 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258976    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 19:25:48.259331 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.258982    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 19:25:48.259948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.259932    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 19:25:48.259948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.259943    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 19:25:48.263603 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.263572    2579 server.go:1268] "Started kubelet"
Jul 27 19:25:48.265165 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.264274    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 19:25:48.265165 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.264441    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 19:25:48.265165 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.264537    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 19:25:48.264579 ip-10-0-131-59 systemd[1]: Started Kubernetes Kubelet.
Jul 27 19:25:48.266352 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.266325    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 19:25:48.266735 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.266706    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 19:25:48.266916 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.266895    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-59.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:25:48.268161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.268117    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:25:48.268161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.268131    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:25:48.271279 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.271246    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 19:25:48.273990 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.272799    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-59.ec2.internal.18c63c1186419ca1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-59.ec2.internal,UID:ip-10-0-131-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-59.ec2.internal,},FirstTimestamp:2026-07-27 19:25:48.263537825 +0000 UTC m=+0.354889331,LastTimestamp:2026-07-27 19:25:48.263537825 +0000 UTC m=+0.354889331,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-59.ec2.internal,}"
Jul 27 19:25:48.275963 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.275949    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 19:25:48.276023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.275959    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 19:25:48.276150 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.276043    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:48.276766 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.276644    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 19:25:48.276766 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.276661    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 19:25:48.276913 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.276776    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 19:25:48.276913 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.276790    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 19:25:48.276913 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.276755    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 19:25:48.277023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.276970    2579 factory.go:55] Registering systemd factory
Jul 27 19:25:48.277023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.276986    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 27 19:25:48.277474 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.277459    2579 factory.go:153] Registering CRI-O factory
Jul 27 19:25:48.277593 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.277580    2579 factory.go:223] Registration of the crio container factory successfully
Jul 27 19:25:48.277668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.277654    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 19:25:48.277725 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.277680    2579 factory.go:103] Registering Raw factory
Jul 27 19:25:48.277725 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.277699    2579 manager.go:1196] Started watching for new ooms in manager
Jul 27 19:25:48.278345 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.278284    2579 manager.go:349] Starting recovery of all containers
Jul 27 19:25:48.284840 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.284818    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cbsrs"
Jul 27 19:25:48.286796 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.286755    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:25:48.286796 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.286770    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 19:25:48.289962 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.289784    2579 manager.go:354] Recovery completed
Jul 27 19:25:48.295087 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.295074    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:25:48.297204 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.297187    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:25:48.297315 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.297217    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:25:48.297315 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.297229    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:25:48.297917 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.297900    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 19:25:48.297917 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.297917    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 19:25:48.298022 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.297939    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 19:25:48.298022 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.297944    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:25:48.298728 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.298708    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:25:48.298826 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.298740    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:25:48.298826 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.298756    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:25:48.298826 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.298789    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cbsrs"
Jul 27 19:25:48.300240 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.300227    2579 policy_none.go:49] "None policy: Start"
Jul 27 19:25:48.300317 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.300246    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 19:25:48.300317 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.300259    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 19:25:48.301420 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.301408    2579 policy_none.go:47] "Start"
Jul 27 19:25:48.339932 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.339909    2579 manager.go:337] "Starting Device Plugin manager"
Jul 27 19:25:48.340029 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.339947    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 19:25:48.340029 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.339958    2579 server.go:87] "Starting device plugin registration server"
Jul 27 19:25:48.340282 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.340261    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 19:25:48.340340 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.340282    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 19:25:48.340403 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.340387    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 19:25:48.340503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.340484    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 19:25:48.340503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.340499    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 19:25:48.341086 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.341066    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 19:25:48.341201 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.341114    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:48.444396 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.444327    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:25:48.445673 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.445450    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:25:48.445673 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.445483    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:25:48.445673 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.445494    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:25:48.445673 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.445522    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.454183 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.454165    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.454252 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.454191    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-59.ec2.internal\": node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:48.480891 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.480844    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 19:25:48.482220 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.482198    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 19:25:48.482362 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.482241    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 19:25:48.482362 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.482266    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 19:25:48.482362 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.482308    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 19:25:48.484999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.484972    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:25:48.557466 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.557432    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:48.583345 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.582673    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal"]
Jul 27 19:25:48.583437 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.583389    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:25:48.584484 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.584466    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:25:48.584584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.584504    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:25:48.584584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.584519    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:25:48.585980 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.585962    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:25:48.586128 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.586093    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.586186 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.586144    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:25:48.586781 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.586764    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:25:48.586854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.586785    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:25:48.586854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.586796    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:25:48.586854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.586808    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:25:48.586854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.586812    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:25:48.586854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.586827    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:25:48.588098 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.588081    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.588177 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.588113    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:25:48.588852 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.588838    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:25:48.588948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.588867    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:25:48.588948 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.588882    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:25:48.611229 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.611200    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-59.ec2.internal\" not found" node="ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.616161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.616142    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-59.ec2.internal\" not found" node="ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.658164 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.658129    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:48.680305 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.680241    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c244d0c96e0f692212c08c924c07be44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal\" (UID: \"c244d0c96e0f692212c08c924c07be44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.680462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.680315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c244d0c96e0f692212c08c924c07be44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal\" (UID: \"c244d0c96e0f692212c08c924c07be44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.680462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.680337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/94b0f96561dcf348510b71c7e517149a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-59.ec2.internal\" (UID: \"94b0f96561dcf348510b71c7e517149a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.758958 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.758888    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:48.781334 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.781303    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c244d0c96e0f692212c08c924c07be44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal\" (UID: \"c244d0c96e0f692212c08c924c07be44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.781468 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.781334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c244d0c96e0f692212c08c924c07be44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal\" (UID: \"c244d0c96e0f692212c08c924c07be44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.781468 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.781352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c244d0c96e0f692212c08c924c07be44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal\" (UID: \"c244d0c96e0f692212c08c924c07be44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.781468 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.781386    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c244d0c96e0f692212c08c924c07be44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal\" (UID: \"c244d0c96e0f692212c08c924c07be44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.781468 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.781397    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/94b0f96561dcf348510b71c7e517149a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-59.ec2.internal\" (UID: \"94b0f96561dcf348510b71c7e517149a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.781468 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.781443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/94b0f96561dcf348510b71c7e517149a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-59.ec2.internal\" (UID: \"94b0f96561dcf348510b71c7e517149a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.862208 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.862158    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:48.916878 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.916846    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.919463 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:48.919442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:48.963126 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:48.963091    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.065952 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:49.065841    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.172476 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:49.172434    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.206437 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.206399    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 19:25:49.207000 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.206564    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 19:25:49.268341 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.268305    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:25:49.273118 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:49.273097    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.303375 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.303327    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 19:20:49 +0000 UTC" deadline="2028-01-30 12:13:11.317110924 +0000 UTC"
Jul 27 19:25:49.303375 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.303370    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13240h47m22.013745769s"
Jul 27 19:25:49.373973 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:49.373942    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.443107 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:49.443070    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b0f96561dcf348510b71c7e517149a.slice/crio-9285f485cd57c02dffe6ec84d90105d4013cde3cdedb28001254414245a37ead WatchSource:0}: Error finding container 9285f485cd57c02dffe6ec84d90105d4013cde3cdedb28001254414245a37ead: Status 404 returned error can't find the container with id 9285f485cd57c02dffe6ec84d90105d4013cde3cdedb28001254414245a37ead
Jul 27 19:25:49.443361 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:49.443339    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc244d0c96e0f692212c08c924c07be44.slice/crio-e4126651af761b73c831788b0834e5b8b80f38a52f24509efaf3082106d58e2a WatchSource:0}: Error finding container e4126651af761b73c831788b0834e5b8b80f38a52f24509efaf3082106d58e2a: Status 404 returned error can't find the container with id e4126651af761b73c831788b0834e5b8b80f38a52f24509efaf3082106d58e2a
Jul 27 19:25:49.451573 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.451555    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:25:49.474107 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:49.474072    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.485258 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.485191    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal" event={"ID":"c244d0c96e0f692212c08c924c07be44","Type":"ContainerStarted","Data":"e4126651af761b73c831788b0834e5b8b80f38a52f24509efaf3082106d58e2a"}
Jul 27 19:25:49.486302 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.486261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal" event={"ID":"94b0f96561dcf348510b71c7e517149a","Type":"ContainerStarted","Data":"9285f485cd57c02dffe6ec84d90105d4013cde3cdedb28001254414245a37ead"}
Jul 27 19:25:49.575957 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:49.575902    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.678341 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:49.678231    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.779452 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:49.779415    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-59.ec2.internal\" not found"
Jul 27 19:25:49.825169 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.825142    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:25:49.844817 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.844791    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:25:49.876644 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.876611    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:49.891477 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.891445    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 19:25:49.892683 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.892638    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal"
Jul 27 19:25:49.901682 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:49.901653    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 19:25:50.252764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.252725    2579 apiserver.go:52] "Watching apiserver"
Jul 27 19:25:50.263404 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.263370    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 19:25:50.263898 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.263867    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-wtrmw","kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r","openshift-cluster-node-tuning-operator/tuned-hcc8q","openshift-dns/node-resolver-vzl59","openshift-image-registry/node-ca-spvgm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal","openshift-multus/multus-additional-cni-plugins-hcqwt","kube-system/konnectivity-agent-dx8ff","openshift-multus/multus-q2v4h","openshift-multus/network-metrics-daemon-qc4zf","openshift-network-diagnostics/network-check-target-9km82","openshift-network-operator/iptables-alerter-cscws","openshift-ovn-kubernetes/ovnkube-node-rmqjz"]
Jul 27 19:25:50.266152 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.266125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.268454 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.268187    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.270467 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.270433    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.271123 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.271098    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 19:25:50.271415 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.271393    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 19:25:50.271726 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.271706    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 19:25:50.271900 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.271857    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 19:25:50.272931 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.272907    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.273989 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.273968    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-956lq\""
Jul 27 19:25:50.274530 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.274507    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rd8fr\""
Jul 27 19:25:50.275159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.274780    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 19:25:50.275159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.274781    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 19:25:50.275159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.275139    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 19:25:50.275408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.275222    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 19:25:50.275408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.275374    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 19:25:50.275544 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.275511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.275617 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.275596    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:25:50.275789 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.275601    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qtkg7\""
Jul 27 19:25:50.276318 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.276022    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lqbgk\""
Jul 27 19:25:50.276318 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.276137    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 19:25:50.276857 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.276839    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 19:25:50.277959 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.277938    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.279093 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.279070    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x95fj\""
Jul 27 19:25:50.279311 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.279289    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:25:50.279659 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.279636    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 19:25:50.279932 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.279909    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 19:25:50.280153 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.280136    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:25:50.280715 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.280692    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xgw9f\""
Jul 27 19:25:50.281981 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.281959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.284551 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.284378    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.286002 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.285980    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 19:25:50.286100 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.286041    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 19:25:50.286195 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.286177    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r29b5\""
Jul 27 19:25:50.286738 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.286702    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.286860 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.286761    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:25:50.287018 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.286999    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 19:25:50.287461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.287435    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5jnpg\""
Jul 27 19:25:50.288360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288162    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 19:25:50.288360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-netns\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.288360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-registration-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.288360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-cni-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.288690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-conf-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.288690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-etc-kubernetes\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.288690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv7vb\" (UniqueName: \"kubernetes.io/projected/bcd1a649-a8e2-4657-ba0d-b19301043af8-kube-api-access-bv7vb\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.288690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.288690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysconfig\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.288690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-system-cni-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.288999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-socket-dir-parent\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.288999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-cni-bin\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.288999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.288810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-hostroot\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.289590 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.289570    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 19:25:50.290004 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.289984    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 19:25:50.291559 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.291531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-daemon-config\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.291656 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.291599    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-var-lib-kubelet\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.291656 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.291632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-host\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.291868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.291665    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 19:25:50.291868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.291689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-kubelet\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.291868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.291701    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 19:25:50.291868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.291723    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 19:25:50.292061 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.291972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-multus-certs\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.292061 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3fc4e27-9510-4feb-a0db-a8c1b320db57-konnectivity-ca\") pod \"konnectivity-agent-dx8ff\" (UID: \"b3fc4e27-9510-4feb-a0db-a8c1b320db57\") " pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:25:50.292163 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-modprobe-d\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.292163 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysctl-d\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.292286 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysctl-conf\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.292286 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-systemd\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.292427 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292272    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-hosts-file\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.292481 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcpgr\" (UniqueName: \"kubernetes.io/projected/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-kube-api-access-xcpgr\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.292481 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9138d8cc-fdd4-4883-84be-0600b09f69a8-serviceca\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.292581 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292520    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j89fr\" (UniqueName: \"kubernetes.io/projected/09e9d3cf-99cc-4c17-8b64-1db7729420e7-kube-api-access-j89fr\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.292641 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-kubernetes\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.292641 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-sys\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.292738 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-lib-modules\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.292738 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292695    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 19:25:50.292857 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-cnibin\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.292857 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292795    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-os-release\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.292857 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcd1a649-a8e2-4657-ba0d-b19301043af8-cni-binary-copy\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.293009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.292858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-k8s-cni-cncf-io\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.293081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9138d8cc-fdd4-4883-84be-0600b09f69a8-host\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.293081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/09e9d3cf-99cc-4c17-8b64-1db7729420e7-iptables-alerter-script\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.293191 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293090    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 19:25:50.293191 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-device-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.293397 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-sys-fs\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.293462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l28w9\" (UniqueName: \"kubernetes.io/projected/b7ec533c-6065-4c89-a978-17a9b5faf834-kube-api-access-l28w9\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.293518 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-run\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.293576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293533    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-tuned\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.293576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293547    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-424sf\""
Jul 27 19:25:50.293576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49a19635-139f-4e3f-9a38-ac5c41be4280-tmp\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.293726 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h866j\" (UniqueName: \"kubernetes.io/projected/49a19635-139f-4e3f-9a38-ac5c41be4280-kube-api-access-h866j\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.293726 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293660    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-cni-multus\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.293726 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-tmp-dir\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.293876 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n65vk\" (UniqueName: \"kubernetes.io/projected/9138d8cc-fdd4-4883-84be-0600b09f69a8-kube-api-access-n65vk\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.293876 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293808    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:50.293876 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3fc4e27-9510-4feb-a0db-a8c1b320db57-agent-certs\") pod \"konnectivity-agent-dx8ff\" (UID: \"b3fc4e27-9510-4feb-a0db-a8c1b320db57\") " pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:25:50.294014 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.293981    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09e9d3cf-99cc-4c17-8b64-1db7729420e7-host-slash\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.294014 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.293962    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:25:50.294176 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.294120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-socket-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.294231 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.294175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-etc-selinux\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.294673 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.294563    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 19:25:50.294874 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.294845    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 19:25:50.295179 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.295131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:50.295427 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.295406    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:25:50.312775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.312747    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 19:25:50.378741 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.378714    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 19:25:50.394988 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.394949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-run\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.395180 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.395180 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-systemd-units\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.395180 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395065    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-run\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.395180 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-slash\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.395180 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqzjl\" (UniqueName: \"kubernetes.io/projected/56643b2e-0ea2-498b-aed2-e185018e7028-kube-api-access-hqzjl\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.395464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwwlb\" (UniqueName: \"kubernetes.io/projected/1681bd39-34f5-4469-8ed5-fc75330bad04-kube-api-access-zwwlb\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:50.395464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/185be14c-e7d0-494b-b626-94f381a9ab93-dbus\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.395464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-netns\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-registration-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.395464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395319    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.395464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395326    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-netns\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-log-socket\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.395464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395460    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97hpq\" (UniqueName: \"kubernetes.io/projected/68a01e15-aea1-4950-bd2a-47bac1e23587-kube-api-access-97hpq\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395472    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-registration-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-cni-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-conf-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-conf-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-etc-kubernetes\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-cni-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395705    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-etc-kubernetes\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv7vb\" (UniqueName: \"kubernetes.io/projected/bcd1a649-a8e2-4657-ba0d-b19301043af8-kube-api-access-bv7vb\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395771    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-ovnkube-config\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/185be14c-e7d0-494b-b626-94f381a9ab93-kubelet-config\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-system-cni-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-socket-dir-parent\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-cni-bin\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-hostroot\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.395982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-daemon-config\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-host\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-run-ovn-kubernetes\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396093    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-kubelet\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-multus-certs\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysctl-d\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-systemd\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-cni-bin\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-lib-modules\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-tuned\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49a19635-139f-4e3f-9a38-ac5c41be4280-tmp\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-cnibin\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396313    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-system-cni-dir\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.396597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-hosts-file\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-socket-dir-parent\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcpgr\" (UniqueName: \"kubernetes.io/projected/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-kube-api-access-xcpgr\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396379    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-multus-certs\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9138d8cc-fdd4-4883-84be-0600b09f69a8-serviceca\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396437    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396492    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-run-netns\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396518    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-var-lib-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-cnibin\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcd1a649-a8e2-4657-ba0d-b19301043af8-cni-binary-copy\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-k8s-cni-cncf-io\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l28w9\" (UniqueName: \"kubernetes.io/projected/b7ec533c-6065-4c89-a978-17a9b5faf834-kube-api-access-l28w9\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h866j\" (UniqueName: \"kubernetes.io/projected/49a19635-139f-4e3f-9a38-ac5c41be4280-kube-api-access-h866j\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-system-cni-dir\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:50.397469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-kubelet\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-systemd\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-etc-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396904    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-ovn\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396956    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-cni-bin\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-cni-multus\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcd1a649-a8e2-4657-ba0d-b19301043af8-multus-daemon-config\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397062    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-systemd\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-host\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-tmp-dir\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-kubelet\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-lib-modules\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n65vk\" (UniqueName: \"kubernetes.io/projected/9138d8cc-fdd4-4883-84be-0600b09f69a8-kube-api-access-n65vk\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-var-lib-cni-multus\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397181    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3fc4e27-9510-4feb-a0db-a8c1b320db57-agent-certs\") pod \"konnectivity-agent-dx8ff\" (UID: \"b3fc4e27-9510-4feb-a0db-a8c1b320db57\") " pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysctl-d\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.398248 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397226    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09e9d3cf-99cc-4c17-8b64-1db7729420e7-host-slash\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-socket-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397303    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09e9d3cf-99cc-4c17-8b64-1db7729420e7-host-slash\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-etc-selinux\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397384    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-cni-netd\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397400    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-etc-selinux\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-tmp-dir\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysconfig\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397505    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-socket-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397503    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-os-release\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397487    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysconfig\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.396413    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-hostroot\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397521    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-cnibin\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397936    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-host-run-k8s-cni-cncf-io\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.397548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-ovnkube-script-lib\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.399142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-var-lib-kubelet\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398061    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68a01e15-aea1-4950-bd2a-47bac1e23587-ovn-node-metrics-cert\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3fc4e27-9510-4feb-a0db-a8c1b320db57-konnectivity-ca\") pod \"konnectivity-agent-dx8ff\" (UID: \"b3fc4e27-9510-4feb-a0db-a8c1b320db57\") " pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398133    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcd1a649-a8e2-4657-ba0d-b19301043af8-cni-binary-copy\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398175    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-modprobe-d\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysctl-conf\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398225    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9138d8cc-fdd4-4883-84be-0600b09f69a8-serviceca\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-sys\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-hosts-file\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-var-lib-kubelet\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j89fr\" (UniqueName: \"kubernetes.io/projected/09e9d3cf-99cc-4c17-8b64-1db7729420e7-kube-api-access-j89fr\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398405    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-sys\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-modprobe-d\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-kubernetes\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398412    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-kubernetes\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398471    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-sysctl-conf\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-node-log\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-env-overrides\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.399999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-os-release\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398686    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3fc4e27-9510-4feb-a0db-a8c1b320db57-konnectivity-ca\") pod \"konnectivity-agent-dx8ff\" (UID: \"b3fc4e27-9510-4feb-a0db-a8c1b320db57\") " pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9138d8cc-fdd4-4883-84be-0600b09f69a8-host\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcd1a649-a8e2-4657-ba0d-b19301043af8-os-release\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/09e9d3cf-99cc-4c17-8b64-1db7729420e7-iptables-alerter-script\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9138d8cc-fdd4-4883-84be-0600b09f69a8-host\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-device-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-sys-fs\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398940    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-device-dir\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.398991    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ec533c-6065-4c89-a978-17a9b5faf834-sys-fs\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.401009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.399557    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/09e9d3cf-99cc-4c17-8b64-1db7729420e7-iptables-alerter-script\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.402525 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.402055    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49a19635-139f-4e3f-9a38-ac5c41be4280-tmp\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.402525 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.402236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/49a19635-139f-4e3f-9a38-ac5c41be4280-etc-tuned\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.402525 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.402365    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3fc4e27-9510-4feb-a0db-a8c1b320db57-agent-certs\") pod \"konnectivity-agent-dx8ff\" (UID: \"b3fc4e27-9510-4feb-a0db-a8c1b320db57\") " pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:25:50.412085 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.412028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n65vk\" (UniqueName: \"kubernetes.io/projected/9138d8cc-fdd4-4883-84be-0600b09f69a8-kube-api-access-n65vk\") pod \"node-ca-spvgm\" (UID: \"9138d8cc-fdd4-4883-84be-0600b09f69a8\") " pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.412212 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.412065    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv7vb\" (UniqueName: \"kubernetes.io/projected/bcd1a649-a8e2-4657-ba0d-b19301043af8-kube-api-access-bv7vb\") pod \"multus-q2v4h\" (UID: \"bcd1a649-a8e2-4657-ba0d-b19301043af8\") " pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.412345 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.412304    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcpgr\" (UniqueName: \"kubernetes.io/projected/3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9-kube-api-access-xcpgr\") pod \"node-resolver-vzl59\" (UID: \"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9\") " pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.416540 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.416102    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28w9\" (UniqueName: \"kubernetes.io/projected/b7ec533c-6065-4c89-a978-17a9b5faf834-kube-api-access-l28w9\") pod \"aws-ebs-csi-driver-node-2zp4r\" (UID: \"b7ec533c-6065-4c89-a978-17a9b5faf834\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.416540 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.416462    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j89fr\" (UniqueName: \"kubernetes.io/projected/09e9d3cf-99cc-4c17-8b64-1db7729420e7-kube-api-access-j89fr\") pod \"iptables-alerter-cscws\" (UID: \"09e9d3cf-99cc-4c17-8b64-1db7729420e7\") " pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.418559 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.418506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h866j\" (UniqueName: \"kubernetes.io/projected/49a19635-139f-4e3f-9a38-ac5c41be4280-kube-api-access-h866j\") pod \"tuned-hcc8q\" (UID: \"49a19635-139f-4e3f-9a38-ac5c41be4280\") " pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.419524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.419498    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qkdqd"
Jul 27 19:25:50.428335 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.428311    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:25:50.428503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.428439    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qkdqd"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-log-socket\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97hpq\" (UniqueName: \"kubernetes.io/projected/68a01e15-aea1-4950-bd2a-47bac1e23587-kube-api-access-97hpq\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500083    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-ovnkube-config\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/185be14c-e7d0-494b-b626-94f381a9ab93-kubelet-config\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500137    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-run-ovn-kubernetes\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500142    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-log-socket\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-cnibin\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500262    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/185be14c-e7d0-494b-b626-94f381a9ab93-kubelet-config\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500214    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-run-ovn-kubernetes\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500315    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.500321    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-cnibin\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500339    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-run-netns\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-var-lib-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.500723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500410    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-var-lib-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.500418    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:51.000373803 +0000 UTC m=+3.091725285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-run-netns\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-system-cni-dir\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-kubelet\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-systemd\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-etc-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-ovn\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-kubelet\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-cni-bin\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.500685    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500714    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-cni-netd\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-ovn\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.501755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500768    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-os-release\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-ovnkube-script-lib\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-os-release\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.500842    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:51.00082284 +0000 UTC m=+3.092174334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500685    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-etc-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-ovnkube-config\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-systemd\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68a01e15-aea1-4950-bd2a-47bac1e23587-ovn-node-metrics-cert\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-cni-netd\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-system-cni-dir\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.500975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-cni-bin\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-node-log\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-env-overrides\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.502648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-node-log\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501165    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-systemd-units\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501230    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-slash\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqzjl\" (UniqueName: \"kubernetes.io/projected/56643b2e-0ea2-498b-aed2-e185018e7028-kube-api-access-hqzjl\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwwlb\" (UniqueName: \"kubernetes.io/projected/1681bd39-34f5-4469-8ed5-fc75330bad04-kube-api-access-zwwlb\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-host-slash\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/185be14c-e7d0-494b-b626-94f381a9ab93-dbus\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-ovnkube-script-lib\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501364    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-systemd-units\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501272    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56643b2e-0ea2-498b-aed2-e185018e7028-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68a01e15-aea1-4950-bd2a-47bac1e23587-run-openvswitch\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/185be14c-e7d0-494b-b626-94f381a9ab93-dbus\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68a01e15-aea1-4950-bd2a-47bac1e23587-env-overrides\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.503503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.501681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56643b2e-0ea2-498b-aed2-e185018e7028-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.504222 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.503852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68a01e15-aea1-4950-bd2a-47bac1e23587-ovn-node-metrics-cert\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.517048 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.517013    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:25:50.517048 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.517048    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:25:50.517300 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.517063    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:50.517300 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:50.517147    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:51.017128398 +0000 UTC m=+3.108479878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:50.520239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.520211    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwwlb\" (UniqueName: \"kubernetes.io/projected/1681bd39-34f5-4469-8ed5-fc75330bad04-kube-api-access-zwwlb\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:50.522463 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.522433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97hpq\" (UniqueName: \"kubernetes.io/projected/68a01e15-aea1-4950-bd2a-47bac1e23587-kube-api-access-97hpq\") pod \"ovnkube-node-rmqjz\" (UID: \"68a01e15-aea1-4950-bd2a-47bac1e23587\") " pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.522570 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.522529    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqzjl\" (UniqueName: \"kubernetes.io/projected/56643b2e-0ea2-498b-aed2-e185018e7028-kube-api-access-hqzjl\") pod \"multus-additional-cni-plugins-hcqwt\" (UID: \"56643b2e-0ea2-498b-aed2-e185018e7028\") " pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:50.582446 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.582408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q2v4h"
Jul 27 19:25:50.591450 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.591419    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cscws"
Jul 27 19:25:50.606698 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.606663    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r"
Jul 27 19:25:50.611766 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.611718    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hcc8q"
Jul 27 19:25:50.619712 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.619684    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vzl59"
Jul 27 19:25:50.635257 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.635224    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-spvgm"
Jul 27 19:25:50.643377 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.643343    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:25:50.651318 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.651295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:25:50.659239 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:50.659213    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:25:51.006576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.006544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:51.006750 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.006602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:51.006750 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.006665    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:51.006750 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.006735    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:51.006750 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.006747    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:52.006727915 +0000 UTC m=+4.098079392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:51.006955 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.006786    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:52.006770423 +0000 UTC m=+4.098121919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:51.107493 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.107455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:51.107658 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.107613    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:25:51.107658 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.107635    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:25:51.107658 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.107649    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:51.107775 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.107715    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:52.10769564 +0000 UTC m=+4.199047121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:51.433661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.433579    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 19:20:51 +0000 UTC" deadline="2028-04-07 23:35:40.424152874 +0000 UTC"
Jul 27 19:25:51.433661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.433613    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14884h9m48.990543587s"
Jul 27 19:25:51.471945 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:51.471912    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ec533c_6065_4c89_a978_17a9b5faf834.slice/crio-22e715276c7ff37c64cf38240302cbb07a43834e7c58a395dcfcc7e8230648f9 WatchSource:0}: Error finding container 22e715276c7ff37c64cf38240302cbb07a43834e7c58a395dcfcc7e8230648f9: Status 404 returned error can't find the container with id 22e715276c7ff37c64cf38240302cbb07a43834e7c58a395dcfcc7e8230648f9
Jul 27 19:25:51.473393 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:51.473363    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3fc4e27_9510_4feb_a0db_a8c1b320db57.slice/crio-87cb8a263ff60c754c54458494ddc9a904b52224469b4a3c7251bee29abc87dc WatchSource:0}: Error finding container 87cb8a263ff60c754c54458494ddc9a904b52224469b4a3c7251bee29abc87dc: Status 404 returned error can't find the container with id 87cb8a263ff60c754c54458494ddc9a904b52224469b4a3c7251bee29abc87dc
Jul 27 19:25:51.474450 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:51.474427    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9138d8cc_fdd4_4883_84be_0600b09f69a8.slice/crio-a6fee8d214ed3a0844d8b8fff77cb85b4725be570a86be66139c1617a046c02b WatchSource:0}: Error finding container a6fee8d214ed3a0844d8b8fff77cb85b4725be570a86be66139c1617a046c02b: Status 404 returned error can't find the container with id a6fee8d214ed3a0844d8b8fff77cb85b4725be570a86be66139c1617a046c02b
Jul 27 19:25:51.478601 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:25:51.478574    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a01e15_aea1_4950_bd2a_47bac1e23587.slice/crio-0e817fd4e4294b8f0a0c0dc1b19727a6bc9bbe59214d703a34907372c765ced3 WatchSource:0}: Error finding container 0e817fd4e4294b8f0a0c0dc1b19727a6bc9bbe59214d703a34907372c765ced3: Status 404 returned error can't find the container with id 0e817fd4e4294b8f0a0c0dc1b19727a6bc9bbe59214d703a34907372c765ced3
Jul 27 19:25:51.484375 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.482813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:51.484375 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:51.482970    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:25:51.492652 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.492611    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vzl59" event={"ID":"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9","Type":"ContainerStarted","Data":"a4a9aa13f9c4ce403351fbf77d75d8861d68f741b75cdc40a9d7250f690ac1f5"}
Jul 27 19:25:51.493981 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.493955    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cscws" event={"ID":"09e9d3cf-99cc-4c17-8b64-1db7729420e7","Type":"ContainerStarted","Data":"83f0586b901d190ba7a57537b735351cc7fd2e5de88962351369c5c51de5cb4d"}
Jul 27 19:25:51.495102 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.495075    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hcc8q" event={"ID":"49a19635-139f-4e3f-9a38-ac5c41be4280","Type":"ContainerStarted","Data":"dd133b1e1f55a59e364d5bdcc9d96ff5b79cf72d24278326fcaf805f77f737c5"}
Jul 27 19:25:51.496423 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.496400    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerStarted","Data":"97f0cb1e126c1ecb1c69009dc88694a3c81a037d3f8e6ad88584e170231a8eb7"}
Jul 27 19:25:51.497736 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.497707    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2v4h" event={"ID":"bcd1a649-a8e2-4657-ba0d-b19301043af8","Type":"ContainerStarted","Data":"1b09fc0ce0e922833403d341d77ed21b1082bd666f118bda26dcb02150b60b05"}
Jul 27 19:25:51.499061 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.498966    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"0e817fd4e4294b8f0a0c0dc1b19727a6bc9bbe59214d703a34907372c765ced3"}
Jul 27 19:25:51.500211 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.500163    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-spvgm" event={"ID":"9138d8cc-fdd4-4883-84be-0600b09f69a8","Type":"ContainerStarted","Data":"a6fee8d214ed3a0844d8b8fff77cb85b4725be570a86be66139c1617a046c02b"}
Jul 27 19:25:51.501498 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.501456    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dx8ff" event={"ID":"b3fc4e27-9510-4feb-a0db-a8c1b320db57","Type":"ContainerStarted","Data":"87cb8a263ff60c754c54458494ddc9a904b52224469b4a3c7251bee29abc87dc"}
Jul 27 19:25:51.502798 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:51.502777    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r" event={"ID":"b7ec533c-6065-4c89-a978-17a9b5faf834","Type":"ContainerStarted","Data":"22e715276c7ff37c64cf38240302cbb07a43834e7c58a395dcfcc7e8230648f9"}
Jul 27 19:25:52.015365 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.015059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:52.015560 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.015470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:52.015683 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.015223    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:52.015783 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.015767    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:54.015742917 +0000 UTC m=+6.107094419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:52.016175 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.015659    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:52.016279 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.016225    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:54.016206917 +0000 UTC m=+6.107558400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:52.116964 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.116919    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:52.117161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.117112    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:25:52.117161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.117134    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:25:52.117161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.117148    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:52.117349 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.117210    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:54.117192433 +0000 UTC m=+6.208543915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:52.434599 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.434526    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 19:20:51 +0000 UTC" deadline="2028-03-17 11:00:20.718247606 +0000 UTC"
Jul 27 19:25:52.434599 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.434596    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14367h34m28.283656301s"
Jul 27 19:25:52.483360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.483309    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:52.483550 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.483529    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:25:52.484222 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.484076    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:52.484222 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:52.484180    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:25:52.517928 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.517887    2579 generic.go:356] "Generic (PLEG): container finished" podID="c244d0c96e0f692212c08c924c07be44" containerID="5d1983389ada9cf0f9875014c553089db324daa42b9c34b7aab96f433cffed55" exitCode=0
Jul 27 19:25:52.518226 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.518006    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal" event={"ID":"c244d0c96e0f692212c08c924c07be44","Type":"ContainerDied","Data":"5d1983389ada9cf0f9875014c553089db324daa42b9c34b7aab96f433cffed55"}
Jul 27 19:25:52.539701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:52.539665    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal" event={"ID":"94b0f96561dcf348510b71c7e517149a","Type":"ContainerStarted","Data":"8c5b87520de2cef1382e3fb0e96221ee36a5b92c0a42dd89590470e36efcafd3"}
Jul 27 19:25:53.482924 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:53.482884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:53.483548 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:53.483042    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:25:53.565564 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:53.565463    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal" event={"ID":"c244d0c96e0f692212c08c924c07be44","Type":"ContainerStarted","Data":"c4b6695d3a8835b1331198f891564b6f1fa8f0598e1054685b65d13a0d932944"}
Jul 27 19:25:53.585625 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:53.585563    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-59.ec2.internal" podStartSLOduration=3.5855418439999998 podStartE2EDuration="3.585541844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:25:52.559629143 +0000 UTC m=+4.650980646" watchObservedRunningTime="2026-07-27 19:25:53.585541844 +0000 UTC m=+5.676893347"
Jul 27 19:25:54.038055 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:54.038009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:54.038301 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:54.038085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:54.038301 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.038272    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:54.038455 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.038364    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:58.038341534 +0000 UTC m=+10.129693018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:54.038868 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.038844    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:54.038948 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.038908    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:58.038891788 +0000 UTC m=+10.130243296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:54.139442 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:54.139380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:54.139650 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.139633    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:25:54.139728 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.139662    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:25:54.139728 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.139705    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:54.139845 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.139767    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:25:58.139748219 +0000 UTC m=+10.231099700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:54.486002 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:54.485919    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:54.486488 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.486073    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:25:54.486488 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:54.486148    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:54.486488 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:54.486227    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:25:55.483545 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:55.483510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:55.483753 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:55.483656    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:25:56.483586 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:56.483284    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:56.483586 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:56.483319    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:56.483586 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:56.483439    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:25:56.483586 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:56.483540    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:25:57.483261 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:57.483222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:57.483456 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:57.483370    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:25:58.075887 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:58.075710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:58.075887 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:58.075806    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:58.076474 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.075914    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:58.076474 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.075968    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:58.076474 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.076004    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:06.075980717 +0000 UTC m=+18.167332222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:25:58.076474 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.076028    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:06.076016248 +0000 UTC m=+18.167367732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:25:58.177260 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:58.177214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:58.177479 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.177446    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:25:58.177479 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.177479    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:25:58.177653 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.177494    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:58.177653 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.177567    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:06.177544247 +0000 UTC m=+18.268895729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:25:58.484030 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:58.483951    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:25:58.484161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.484084    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:25:58.484487 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:58.484460    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:25:58.484631 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:58.484607    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:25:59.483235 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:25:59.483198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:25:59.483758 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:25:59.483358    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:00.483789 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:00.483516    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:00.483789 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:00.483546    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:00.483789 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:00.483693    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:00.484351 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:00.483799    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:01.483360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:01.483329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:01.483541 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:01.483425    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:02.483455 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:02.483394    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:02.483917 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:02.483555    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:02.483917 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:02.483590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:02.483917 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:02.483703    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:03.482662 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:03.482630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:03.482853 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:03.482750    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:04.482924 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:04.482888    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:04.482924 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:04.482937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:04.483447 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:04.483057    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:04.483447 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:04.483179    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:05.482886 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:05.482851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:05.483075 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:05.483018    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:06.136023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:06.135983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:06.136218 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:06.136089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:06.136218 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.136155    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:26:06.136218 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.136205    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:26:06.136347 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.136252    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:22.136221999 +0000 UTC m=+34.227573478 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:26:06.136347 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.136271    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:22.136264588 +0000 UTC m=+34.227616065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:26:06.237190 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:06.237153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:06.237362 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.237293    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:26:06.237362 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.237322    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:26:06.237362 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.237336    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:26:06.237506 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.237394    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:22.237379228 +0000 UTC m=+34.328730713 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:26:06.482722 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:06.482685    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:06.482908 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:06.482685    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:06.482908 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.482830    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:06.483012 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:06.482977    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:07.482808 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:07.482769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:07.483225 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:07.482881    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:08.483461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:08.483421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:08.483899 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:08.483560    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:08.483899 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:08.483613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:08.483899 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:08.483728    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:09.482918 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:09.482864    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:09.483102 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:09.483013    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:10.486364 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:10.486332    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:10.486925 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:10.486458    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:10.486925 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:10.486331    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:10.486925 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:10.486551    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:11.483639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.483179    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:11.483895 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:11.483746    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:11.607868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.607828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vzl59" event={"ID":"3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9","Type":"ContainerStarted","Data":"0834e174088aff3b1022f78ed92a973dc737d45e909c9bdcd9d06e49e38cacd4"}
Jul 27 19:26:11.609179 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.609155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hcc8q" event={"ID":"49a19635-139f-4e3f-9a38-ac5c41be4280","Type":"ContainerStarted","Data":"3e16483a35032dc3466b3a02f71d4194c293c38b6a05e7f68bdcc7e8ea0eaae7"}
Jul 27 19:26:11.610672 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.610646    2579 generic.go:356] "Generic (PLEG): container finished" podID="56643b2e-0ea2-498b-aed2-e185018e7028" containerID="648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42" exitCode=0
Jul 27 19:26:11.610770 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.610731    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerDied","Data":"648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42"}
Jul 27 19:26:11.612317 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.612272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2v4h" event={"ID":"bcd1a649-a8e2-4657-ba0d-b19301043af8","Type":"ContainerStarted","Data":"17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d"}
Jul 27 19:26:11.615060 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.615028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"dc6db2c752d35e3d7a43c5ed94803dc3a3a192fda87035932d3dbbbdc8f9fda1"}
Jul 27 19:26:11.615142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.615091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"f934629c304dec56b6341a0d62b35f7d066b0c3c691189ce61cb632ff6355ccd"}
Jul 27 19:26:11.615185 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.615139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"bf555d82605764ea5d641a51b2077f8d38a5b541372a235c743895b1a2a676ec"}
Jul 27 19:26:11.615185 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.615178    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"29fb8aad8138d6751c5478b6e6ee863d490cd5d96c5daf833ffd482911702ca7"}
Jul 27 19:26:11.615305 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.615207    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"cf429e393aca3c0ea22a832074b0a382e4c67432b08543028e41e5f1063bf452"}
Jul 27 19:26:11.615305 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.615268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"dbf59404bbabd9ff4b2518939d7428eb39da70dbd6816197270bd53f3aad9861"}
Jul 27 19:26:11.616366 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.616347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-spvgm" event={"ID":"9138d8cc-fdd4-4883-84be-0600b09f69a8","Type":"ContainerStarted","Data":"517eaf629b607011f14bbda33c52555b633849a141b0def66ea94573a19417b9"}
Jul 27 19:26:11.617632 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.617612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dx8ff" event={"ID":"b3fc4e27-9510-4feb-a0db-a8c1b320db57","Type":"ContainerStarted","Data":"d2d20c5ecc85d78f1d00d2a77bd554394b0f9f3b747534de5566f86bfa979752"}
Jul 27 19:26:11.618997 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.618979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r" event={"ID":"b7ec533c-6065-4c89-a978-17a9b5faf834","Type":"ContainerStarted","Data":"9a70dab466b8df7cac9f5d7407eab415e95273f227889f41efe73803b73feb52"}
Jul 27 19:26:11.626645 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.626611    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-59.ec2.internal" podStartSLOduration=21.626599205 podStartE2EDuration="21.626599205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:25:53.586961824 +0000 UTC m=+5.678313328" watchObservedRunningTime="2026-07-27 19:26:11.626599205 +0000 UTC m=+23.717950702"
Jul 27 19:26:11.660860 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.660807    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vzl59" podStartSLOduration=3.580902825 podStartE2EDuration="22.660792872s" totalImagesPullingTime="19.079890047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:11.627852545 +0000 UTC m=+23.719204044" watchObservedRunningTime="2026-07-27 19:26:11.660792872 +0000 UTC m=+23.752144423"
Jul 27 19:26:11.680115 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.680066    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dx8ff" podStartSLOduration=11.216065622 podStartE2EDuration="22.680050816s" totalImagesPullingTime="11.463985194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:11.679998038 +0000 UTC m=+23.771349535" watchObservedRunningTime="2026-07-27 19:26:11.680050816 +0000 UTC m=+23.771402313"
Jul 27 19:26:11.680309 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.680199    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q2v4h" podStartSLOduration=3.532243318 podStartE2EDuration="22.680189538s" totalImagesPullingTime="19.14794622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:11.660931249 +0000 UTC m=+23.752282725" watchObservedRunningTime="2026-07-27 19:26:11.680189538 +0000 UTC m=+23.771541036"
Jul 27 19:26:11.759298 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:11.759202    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hcc8q" podStartSLOduration=3.673525388 podStartE2EDuration="22.759186999s" totalImagesPullingTime="19.085661611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:11.737973019 +0000 UTC m=+23.829324516" watchObservedRunningTime="2026-07-27 19:26:11.759186999 +0000 UTC m=+23.850538497"
Jul 27 19:26:12.133759 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.133731    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 19:26:12.353534 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.353359    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T19:26:12.133752776Z","UUID":"5cea3e11-295f-4054-85ea-a59d4155b9a5","Handler":null,"Name":"","Endpoint":""}
Jul 27 19:26:12.354983 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.354960    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 19:26:12.355097 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.355001    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 19:26:12.488920 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.488882    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:12.489107 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:12.489002    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:12.489107 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.489046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:12.489206 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:12.489159    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:12.624545 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.624417    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r" event={"ID":"b7ec533c-6065-4c89-a978-17a9b5faf834","Type":"ContainerStarted","Data":"fb8bbfd3b5b8e53a4893eda67570778160d1e4b3892eb50d84b0bce1362fce03"}
Jul 27 19:26:12.626990 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.626959    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cscws" event={"ID":"09e9d3cf-99cc-4c17-8b64-1db7729420e7","Type":"ContainerStarted","Data":"6215e3306eb60305aa9f35d481e42b1d08930dc6172bf09f1d36dc108e9bba56"}
Jul 27 19:26:12.643689 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:12.643620    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-spvgm" podStartSLOduration=4.549658544 podStartE2EDuration="23.643600942s" totalImagesPullingTime="19.093942398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:11.758939784 +0000 UTC m=+23.850291279" watchObservedRunningTime="2026-07-27 19:26:12.643600942 +0000 UTC m=+24.734952441"
Jul 27 19:26:13.483260 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:13.483046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:13.483436 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:13.483355    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:13.632885 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:13.632845    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"a7d254f33d22b5634fd96a603f3656601baff0c9fcf80f72757088742b23cf0f"}
Jul 27 19:26:13.635555 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:13.635520    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r" event={"ID":"b7ec533c-6065-4c89-a978-17a9b5faf834","Type":"ContainerStarted","Data":"9e8208515b4a5264239d9b94b8a171dbc00ef2ee6b63a931bab417731d1400ed"}
Jul 27 19:26:13.655358 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:13.655297    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cscws" podStartSLOduration=5.574450212 podStartE2EDuration="24.655274584s" totalImagesPullingTime="19.080824372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:12.643368231 +0000 UTC m=+24.734719730" watchObservedRunningTime="2026-07-27 19:26:13.655274584 +0000 UTC m=+25.746626082"
Jul 27 19:26:13.655661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:13.655630    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2zp4r" podStartSLOduration=3.32753971 podStartE2EDuration="24.655620001s" totalImagesPullingTime="21.328080291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:13.655060203 +0000 UTC m=+25.746411701" watchObservedRunningTime="2026-07-27 19:26:13.655620001 +0000 UTC m=+25.746971500"
Jul 27 19:26:14.482901 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:14.482862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:14.482901 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:14.482916    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:14.483161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:14.483024    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:14.483161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:14.483105    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:15.482671 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.482630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:15.483237 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:15.482750    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:15.644754 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.644626    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b"}
Jul 27 19:26:15.645211 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.645060    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:26:15.647035 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.645961    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:26:15.647035 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.646036    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:26:15.665501 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.664952    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:26:15.665501 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.665361    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:26:15.676053 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.675201    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" podStartSLOduration=7.521199251 podStartE2EDuration="26.675183918s" totalImagesPullingTime="19.153984667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:15.673727012 +0000 UTC m=+27.765078510" watchObservedRunningTime="2026-07-27 19:26:15.675183918 +0000 UTC m=+27.766535417"
Jul 27 19:26:15.777101 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.776838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:15.896274 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:15.896157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:16.024288 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.024257    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:26:16.024791 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.024771    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:26:16.307451 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.307418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:16.483048 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.482953    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:16.483865 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:16.483063    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:16.483865 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.483119    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:16.483865 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:16.483208    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:16.649046 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.649006    2579 generic.go:356] "Generic (PLEG): container finished" podID="56643b2e-0ea2-498b-aed2-e185018e7028" containerID="e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f" exitCode=0
Jul 27 19:26:16.649199 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.649086    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerDied","Data":"e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f"}
Jul 27 19:26:16.649291 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.649274    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:26:16.649748 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:16.649727    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dx8ff"
Jul 27 19:26:17.482634 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:17.482598    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:17.482812 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:17.482707    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:17.544901 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:17.544864    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:18.483199 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:18.483159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:18.483363 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:18.483307    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:18.483401 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:18.483359    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:18.483475 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:18.483457    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:18.655700 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:18.655667    2579 generic.go:356] "Generic (PLEG): container finished" podID="56643b2e-0ea2-498b-aed2-e185018e7028" containerID="bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c" exitCode=0
Jul 27 19:26:18.656289 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:18.655745    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerDied","Data":"bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c"}
Jul 27 19:26:19.248720 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:19.248671    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:19.269523 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:19.269447    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:19.327850 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:19.327042    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:19.342767 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:19.342674    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:19.358113 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:19.358017    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:19.371899 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:19.371828    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:19Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:19.371899 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:19.371895    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:26:19.483403 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:19.483376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:19.483594 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:19.483486    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:19.679161 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:19.679055    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"message\\\":\\\"containers with incomplete status: [routeoverride-cni whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"type\\\":\\\"Initialized\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:19.819286 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:19.819211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:20.483490 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:20.483455    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:20.483641 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:20.483596    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:20.483695 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:20.483649    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:20.483795 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:20.483777    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:20.662963 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:20.662929    2579 generic.go:356] "Generic (PLEG): container finished" podID="56643b2e-0ea2-498b-aed2-e185018e7028" containerID="4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00" exitCode=0
Jul 27 19:26:20.663116 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:20.662997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerDied","Data":"4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00"}
Jul 27 19:26:20.683112 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:20.683011    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"message\\\":\\\"containers with incomplete status: [whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"type\\\":\\\"Initialized\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:21.483333 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:21.483295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:21.483512 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:21.483433    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:22.150828 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:22.150788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:22.151377 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:22.150857    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:22.151377 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.150949    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:26:22.151377 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.150969    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:26:22.151377 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.151035    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:54.151014925 +0000 UTC m=+66.242366405 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:26:22.151377 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.151058    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:54.151047678 +0000 UTC m=+66.242399159 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:26:22.251669 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:22.251633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:22.251853 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.251804    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:26:22.251853 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.251827    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:26:22.251853 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.251840    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:26:22.251974 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.251905    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:26:54.251886443 +0000 UTC m=+66.343237921 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:26:22.379545 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:22.379500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:22.482997 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:22.482956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:22.482997 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:22.482985    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:22.483259 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.483089    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:22.483259 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:22.483203    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:22.948253 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:22.948200    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:23.483212 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:23.483175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:23.483666 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:23.483300    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:23.575388 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:23.575354    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:24.487942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:24.487915    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:24.488343 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:24.487980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:24.488343 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:24.488083    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:24.488343 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:24.488183    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:24.676063 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:24.675867    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerStarted","Data":"de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df"}
Jul 27 19:26:24.696687 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:24.696578    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"message\\\":\\\"containers with incomplete status: [whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"type\\\":\\\"Initialized\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:25.482695 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:25.482664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:25.482873 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:25.482782    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:25.681087 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:25.681049    2579 generic.go:356] "Generic (PLEG): container finished" podID="56643b2e-0ea2-498b-aed2-e185018e7028" containerID="de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df" exitCode=0
Jul 27 19:26:25.681465 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:25.681128    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerDied","Data":"de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df"}
Jul 27 19:26:25.701911 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:25.701806    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"message\\\":\\\"containers with incomplete status: [whereabouts-cni]\\\",\\\"type\\\":\\\"Initialized\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:26.482722 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:26.482687    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:26.482889 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:26.482789    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:26.482889 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:26.482837    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:26.482959 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:26.482926    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:26.685582 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:26.685547    2579 generic.go:356] "Generic (PLEG): container finished" podID="56643b2e-0ea2-498b-aed2-e185018e7028" containerID="d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129" exitCode=0
Jul 27 19:26:26.685582 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:26.685585    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerDied","Data":"d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129"}
Jul 27 19:26:26.702717 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:26.702588    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:27.362515 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:27.362480    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:27.483439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:27.483408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:27.483606 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:27.483520    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:27.691525 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:27.691486    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" event={"ID":"56643b2e-0ea2-498b-aed2-e185018e7028","Type":"ContainerStarted","Data":"bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445"}
Jul 27 19:26:27.708424 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:27.708306    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:28.486820 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:28.486791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:28.487009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:28.486791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:28.487009 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:28.486915    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:28.487009 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:28.486966    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:28.500747 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:28.500634    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:29.483209 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:29.483176    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:29.483617 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:29.483309    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:29.565686 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:29.565610    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:29.577769 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:29.577695    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:29.588711 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:29.588639    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:29.599781 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:29.599711    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:29.611153 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:29.611078    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:29Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:29.611153 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:29.611149    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:26:30.483236 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:30.483201    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:30.483668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:30.483203    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:30.483668 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:30.483332    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:30.483668 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:30.483415    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:31.180443 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:31.180406    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:31.296870 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:31.296839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:31.483323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:31.483223    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:31.483760 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:31.483367    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:32.483336 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:32.483301    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:32.483822 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:32.483401    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:32.483822 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:32.483431    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:32.483822 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:32.483487    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:33.482518 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:33.482489    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:33.482689 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:33.482585    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:34.486033 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:34.486007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:34.486477 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:34.486100    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:34.486477 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:34.486177    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:34.486477 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:34.486279    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:35.482692 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:35.482655    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:35.482894 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:35.482772    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:36.200420 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:36.200376    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:36.283707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:36.283672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:36.482721 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:36.482635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:36.482875 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:36.482635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:36.482875 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:36.482765    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:36.482875 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:36.482813    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:37.483023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:37.482987    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:37.483493 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:37.483110    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:38.483541 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:38.483502    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:38.483541 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:38.483544    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:38.484041 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:38.483629    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:38.484041 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:38.483728    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:38.499641 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:38.499507    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:39.482718 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:39.482682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:39.482890 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:39.482780    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:39.724800 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:39.724712    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:39.736917 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:39.736768    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:39.748475 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:39.748399    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:39.761095 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:39.761003    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:39.774105 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:39.774026    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:39Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:39.774105 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:39.774101    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:26:40.483051 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:40.483015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:40.483225 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:40.483021    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:40.483225 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:40.483123    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:40.483318 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:40.483224    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:40.807870 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:40.807784    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:41.194008 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:41.193973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:41.297890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:41.297850    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:41.414061 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:41.414028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:41.483394 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:41.483322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:41.483578 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:41.483443    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:42.482806 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:42.482777    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:42.483292 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:42.482777    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:42.483292 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:42.482922    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:42.483292 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:42.482993    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:43.482670 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:43.482630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:43.482849 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:43.482731    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:43.684777 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:43.684745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:44.261538 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:44.261504    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:44.482807 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:44.482772    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:44.482997 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:44.482774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:44.482997 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:44.482880    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:44.482997 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:44.482955    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:45.482435 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:45.482396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:45.482612 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:45.482497    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:46.483215 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:46.483182    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:46.483728 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:46.483296    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:46.483728 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:46.483370    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:46.483728 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:46.483458    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:47.483180 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:47.483131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:47.483410 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:47.483288    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:47.661228 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:47.661183    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" podUID="68a01e15-aea1-4950-bd2a-47bac1e23587" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 27 19:26:48.483923 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:48.483889    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:48.484465 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:48.484001    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:48.484465 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:48.484031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:48.484465 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:48.484079    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:48.500833 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:48.500719    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:49.482779 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:49.482738    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:49.482983 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:49.482862    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:50.122784 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:50.122701    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:50.176044 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:50.175957    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:50.198936 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:50.198855    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:50.214825 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:50.214739    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:50.228891 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:50.228809    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:26:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:26:50.228891 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:50.228886    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:26:50.483081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:50.483048    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:50.483296 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:50.483190    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:50.483296 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:50.483233    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:50.483428 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:50.483342    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:51.483302 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:51.483267    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:51.483695 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:51.483370    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:52.483016 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:52.482974    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:52.483222 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:52.483027    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:52.483222 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:52.483124    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:52.483358 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:52.483272    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:53.482522 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:53.482493    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:53.482687 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:53.482608    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:54.059699 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:54.059657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:54.176009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:54.175969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:54.176009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:54.176012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:54.176200 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.176112    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:26:54.176200 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.176161    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:58.176146507 +0000 UTC m=+130.267497982 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:26:54.176200 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.176112    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:26:54.176323 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.176259    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:58.176226806 +0000 UTC m=+130.267578296 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:26:54.276631 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:54.276590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:54.276810 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.276742    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:26:54.276810 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.276761    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:26:54.276810 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.276773    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:26:54.276992 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.276834    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:58.276817021 +0000 UTC m=+130.368168521 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:26:54.482975 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:54.482947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:54.483148 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.483059    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:54.483148 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:54.483114    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:54.483280 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:54.483259    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:54.626035 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:54.625998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:55.482599 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:55.482563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:55.482970 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:55.482673    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:56.483343 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:56.483307    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:56.483897 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:56.483459    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:56.483897 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:56.483509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:56.483897 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:56.483624    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:56.760144 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:56.760074    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/0.log"
Jul 27 19:26:56.760144 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:56.760112    2579 generic.go:356] "Generic (PLEG): container finished" podID="bcd1a649-a8e2-4657-ba0d-b19301043af8" containerID="17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d" exitCode=1
Jul 27 19:26:56.760329 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:56.760160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2v4h" event={"ID":"bcd1a649-a8e2-4657-ba0d-b19301043af8","Type":"ContainerDied","Data":"17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d"}
Jul 27 19:26:56.760550 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:56.760525    2579 scope.go:117] "RemoveContainer" containerID="17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d"
Jul 27 19:26:56.777625 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:56.777561    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:56Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:56Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:26:56.790728 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:56.790551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:56.792720 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:56.792549    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:57.483403 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:57.483358    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:57.483826 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:57.483473    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:26:57.763660 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:57.763587    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/0.log"
Jul 27 19:26:57.763799 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:57.763655    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2v4h" event={"ID":"bcd1a649-a8e2-4657-ba0d-b19301043af8","Type":"ContainerStarted","Data":"507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224"}
Jul 27 19:26:57.777306 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:57.777231    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:26:57.790629 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:57.790522    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:58.418319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:58.418286    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:26:58.486275 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:58.486237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:26:58.486635 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:58.486234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:26:58.486635 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:58.486346    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:26:58.486635 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:58.486412    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:26:58.497502 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:58.497440    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:26:58.510688 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:58.510569    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:26:59.483433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:26:59.483401    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:26:59.483702 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:26:59.483501    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:00.483439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:00.483400    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:00.483938 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:00.483501    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:00.483938 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:00.483576    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:00.483938 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:00.483663    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:00.619363 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:00.619280    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:00.630424 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:00.630340    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:00.641794 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:00.641714    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:00.654145 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:00.654061    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:00.665669 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:00.665586    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:00Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:00.665669 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:00.665661    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:27:01.483393 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:01.483358    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:01.483602 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:01.483478    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:02.482701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:02.482664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:02.482701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:02.482682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:02.482962 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:02.482770    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:02.482962 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:02.482834    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:03.482977 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:03.482941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:03.483399 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:03.483037    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:03.933367 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:03.933332    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:04.482712 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:04.482676    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:04.482885 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:04.482686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:04.482885 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:04.482791    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:04.482956 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:04.482886    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:05.482707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:05.482660    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:05.483103 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:05.482769    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:06.483439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:06.483402    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:06.483869 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:06.483404    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:06.483869 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:06.483520    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:06.483869 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:06.483618    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:07.483252 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:07.483220    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:07.483434 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:07.483328    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:08.483463 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:08.483411    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:08.483463 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:08.483453    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:08.483861 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:08.483534    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:08.483861 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:08.483647    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:08.497636 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:08.497576    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:08.512302 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:08.512090    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:09.483269 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:09.483195    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:09.483437 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:09.483355    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:10.483262 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:10.483222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:10.483262 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:10.483233    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:10.483770 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:10.483364    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:10.483770 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:10.483507    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:10.961751 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:10.961665    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:10.972743 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:10.972667    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:10.985209 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:10.985124    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:10.997776 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:10.997697    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:10Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:11.013111 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:11.013017    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:11Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:11Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:11Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:11Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:11.013111 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:11.013099    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:27:11.483097 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:11.483063    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:11.483329 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:11.483167    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:12.483264 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:12.483211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:12.483264 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:12.483211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:12.483696 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:12.483523    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:12.483696 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:12.483606    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:13.483182 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:13.483154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:13.483373 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:13.483264    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:13.906832 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:13.906744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:14.483351 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:14.483315    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:14.483534 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:14.483316    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:14.483534 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:14.483413    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:14.483620 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:14.483545    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:15.226593 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:15.226548    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:15.483465 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:15.483367    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:15.483623 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:15.483483    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:16.482652 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:16.482617    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:16.483141 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:16.482785    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:16.483141 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:16.482822    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:16.483141 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:16.482893    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:17.483450 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:17.483415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:17.483860 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:17.483524    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:17.660577 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:17.660541    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" podUID="68a01e15-aea1-4950-bd2a-47bac1e23587" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 27 19:27:18.483121 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:18.483088    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:18.483308 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:18.483178    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:18.483308 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:18.483212    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:18.483308 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:18.483264    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:18.492778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:18.492747    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" podUID="68a01e15-aea1-4950-bd2a-47bac1e23587" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 27 19:27:18.498399 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:18.498346    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:18.512040 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:18.511932    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:19.482494 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:19.482460    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:19.482645 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:19.482560    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:20.482823 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:20.482790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:20.483213 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:20.482893    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:20.483213 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:20.482965    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:20.483213 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:20.483044    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:21.304649 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:21.304570    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:21.316379 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:21.316284    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:21.328539 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:21.328459    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:21.340011 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:21.339931    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:21.351299 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:21.351203    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:21Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:21.351299 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:21.351294    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:27:21.449555 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:21.449509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:21.482649 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:21.482619    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:21.482851 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:21.482824    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:22.483025 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:22.482988    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:22.483531 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:22.482988    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:22.483531 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:22.483100    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:22.483531 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:22.483157    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:23.483143 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:23.483096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:23.483561 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:23.483327    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:24.482931 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:24.482896    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:24.483098 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:24.482899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:24.483098 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:24.483025    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:24.483173 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:24.483144    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:25.212352 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:25.212313    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:25.483501 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:25.483407    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:25.483952 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:25.483620    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:26.483217 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:26.483180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:26.483413 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:26.483180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:26.483413 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:26.483327    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:26.483413 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:26.483360    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:27.482994 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:27.482966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:27.483391 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:27.483068    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:27.597022 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:27.596987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:28.484282 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:28.484237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:28.484664 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:28.484370    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:28.484664 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:28.484421    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:28.484664 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:28.484638    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:28.498095 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:28.498033    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:28.513667 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:28.513553    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:28.672338 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:28.672305    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:29.482504 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:29.482468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:29.482682 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:29.482572    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:29.900257 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:29.900157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:30.483498 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:30.483465    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:30.483498 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:30.483461    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:30.483820 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:30.483726    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:30.483882 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:30.483846    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:31.483166 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:31.483135    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:31.483559 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:31.483239    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:31.540605 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:31.540523    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:31.551724 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:31.551644    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:31.563339 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:31.563254    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:31.574207 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:31.574122    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:31.585669 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:31.585585    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:31Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:31.585669 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:31.585659    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:27:32.482712 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:32.482672    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:32.482914 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:32.482681    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:32.482914 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:32.482814    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:32.482914 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:32.482875    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:33.279943 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:33.279903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:33.483396 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:33.483359    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:33.483596 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:33.483487    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:34.483218 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:34.483184    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:34.483625 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:34.483193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:34.483625 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:34.483316    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:34.483625 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:34.483358    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:35.005973 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:35.005940    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:35.483070 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:35.483036    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:35.483271 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:35.483138    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:36.052028 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:36.051988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:36.483451 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:36.483417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:36.483860 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:36.483416    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:36.483860 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:36.483526    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:36.483860 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:36.483631    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:37.483277 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:37.483234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:37.483454 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:37.483350    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:38.383851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:38.383816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:38.483943 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:38.483902    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:38.484117 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:38.484006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:38.484726 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:38.484370    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:38.484726 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:38.484644    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:38.499487 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:38.499370    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:38.512023 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:38.511958    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:39.482945 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:39.482909    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:39.483432 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:39.483131    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:40.482666 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:40.482627    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:40.482869 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:40.482742    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:40.482869 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:40.482790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:40.482984 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:40.482881    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:40.565596 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:40.565551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:40.669844 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:40.669797    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:40.771327 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:40.771230    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:40.868736 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:40.868702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:40.961368 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:40.961329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:41.060850 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:41.060765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:41.482755 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:41.482726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:41.482921 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.482826    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:41.606643 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.606559    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:41.618257 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.618161    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:41.630545 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.630462    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:41.642120 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.642038    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:41.653442 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.653358    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\"],\\\"sizeBytes\\\":1341984117},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\"],\\\"sizeBytes\\\":1183199980},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0663b35dc117c67b903f052040d0ce01685d1e22781bc2ef837fb32d942f677c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:89bbd9284fb92be98f07e3067b3dd3b08642713706cd6520e6ed5a7819c4d18e\\\"],\\\"sizeBytes\\\":876273280},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73a0d41e53dabfc205782cf08f95c47e93f3857314598fbfd7b4d281b4a932b2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811\\\"],\\\"sizeBytes\\\":586731180},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\"],\\\"sizeBytes\\\":580608166},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8fe761810ead4093b28f3e7ee9ef6b962fd3499250b1838a1b00cc77765d1bf\\\"],\\\"sizeBytes\\\":537939273},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3ba88cacf3c747ed1559bdb550d4483a0e7a25ad4f76ef3c4588f72ae1d281e\\\"],\\\"sizeBytes\\\":349477060},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf790441537ab71f6e39920d97984a14115bd52c94cb371ffd5cf4ef6899e344\\\"],\\\"sizeBytes\\\":347020814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6db0ab20c9e44aed5dff7711ce889fe6725a09b24499fc7971edcd307d1e942\\\"],\\\"sizeBytes\\\":343989083},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:44dcc3802076a6fb1ea11dff5479cc5721d7c3452fb04af6023d965e5d92c987\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\\\"],\\\"sizeBytes\\\":317186379},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:843b51893f8a3884f1fe56948902e4dd166193b21cc34e099aa8d96a8ff2ea09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f01dca26528042bc3f05fdd53d5a72cb9f3bd70dcfe2f1d5eab626c7530dd859\\\"],\\\"sizeBytes\\\":313107438},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46a09b61078082511645fb4b54f513d0b4c65ffe5abdd774765376c5b5df57e7\\\"],\\\"sizeBytes\\\":284044146},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\"],\\\"sizeBytes\\\":278280461},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f984df1cb2384583cc1b1d74cc304af86ed19d269145c54f380ee1b6b11ebeff\\\"],\\\"sizeBytes\\\":235560681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e0fbdb86532d771d5307b5972b2013a378dfa3a213b2f5960eece8410c770ca3\\\"],\\\"sizeBytes\\\":235302136},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15ab63548751bddcfa77c870dddc6d9bb822b23f82cba324e5f1747c0d35abe8\\\"],\\\"sizeBytes\\\":222771689},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\"],\\\"sizeBytes\\\":218369365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fef1a9d7fb1e554165cc9ff6b4ca01027ee426c0e28b7a9f81ce5339b8bb53f\\\"],\\\"sizeBytes\\\":206446249}]}}\" for node \"ip-10-0-131-59.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/node?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused"
Jul 27 19:27:41.653442 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.653433    2579 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Jul 27 19:27:41.865939 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:41.865859    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/0.log"
Jul 27 19:27:41.867647 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:41.867624    2579 generic.go:356] "Generic (PLEG): container finished" podID="68a01e15-aea1-4950-bd2a-47bac1e23587" containerID="4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b" exitCode=1
Jul 27 19:27:41.867757 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:41.867660    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerDied","Data":"4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b"}
Jul 27 19:27:41.868458 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:41.868444    2579 scope.go:117] "RemoveContainer" containerID="4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b"
Jul 27 19:27:41.887395 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.887223    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"68a01e15-aea1-4950-bd2a-47bac1e23587\\\"},\\\"status\\\":{\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://29fb8aad8138d6751c5478b6e6ee863d490cd5d96c5daf833ffd482911702ca7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://bf555d82605764ea5d641a51b2077f8d38a5b541372a235c743895b1a2a676ec\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dc6db2c752d35e3d7a43c5ed94803dc3a3a192fda87035932d3dbbbdc8f9fda1\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://f934629c304dec56b6341a0d62b35f7d066b0c3c691189ce61cb632ff6355ccd\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://cf429e393aca3c0ea22a832074b0a382e4c67432b08543028e41e5f1063bf452\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dbf59404bbabd9ff4b2518939d7428eb39da70dbd6816197270bd53f3aad9861\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"},\\\"containerID\\\":\\\"cri-o://4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"message\\\":\\\"reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressservice/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:41.231603    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.EgressIP\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressip/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:41.233366    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1alpha1.AdminNetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"sigs.k8s.io/network-policy-api/pkg/client/informers/externalversions/factory.go:141\\\\\\\"\\\\nI0727 19:27:41.233418    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1alpha1.BaselineAdminNetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"sigs.k8s.io/network-policy-api/pkg/client/informers/externalversions/factory.go:141\\\\\\\"\\\\nI0727 19:27:41.236931    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Namespace\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:41.237381    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Service\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:41.237570    3403 factory.go:718] Stopping watch factory\\\\nI0727 19:27:41.237643    3403 ovnkube.go:653] Stopped ovnkube\\\\nI0727 19:27:41.237667    3403 server.go:251] Stopping metrics server at address \\\\\\\"127.0.0.1:29103\\\\\\\"\\\\nF0727 19:27:41.237717    3403 ovnkube.go:149] failed to run ovnkube: [failed to start network controller: failed to start default network controller - while waiting for \\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://a7d254f33d22b5634fd96a603f3656601baff0c9fcf80f72757088742b23cf0f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:13Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-rmqjz\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:27:41.901883 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.901771    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:41.914146 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:41.914069    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:57Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:42.483310 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.483279    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:42.483526 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.483279    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:42.483526 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.483393    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:42.483526 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.483447    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:42.707725 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.707688    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:42.871505 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.871480    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:27:42.871888 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.871875    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/0.log"
Jul 27 19:27:42.871929 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.871905    2579 generic.go:356] "Generic (PLEG): container finished" podID="bcd1a649-a8e2-4657-ba0d-b19301043af8" containerID="507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224" exitCode=1
Jul 27 19:27:42.872001 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.871979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2v4h" event={"ID":"bcd1a649-a8e2-4657-ba0d-b19301043af8","Type":"ContainerDied","Data":"507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224"}
Jul 27 19:27:42.872069 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.872055    2579 scope.go:117] "RemoveContainer" containerID="17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d"
Jul 27 19:27:42.872437 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.872418    2579 scope.go:117] "RemoveContainer" containerID="507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224"
Jul 27 19:27:42.872586 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.872568    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with CrashLoopBackOff: \"back-off 10s restarting failed container=kube-multus pod=multus-q2v4h_openshift-multus(bcd1a649-a8e2-4657-ba0d-b19301043af8)\"" pod="openshift-multus/multus-q2v4h" podUID="bcd1a649-a8e2-4657-ba0d-b19301043af8"
Jul 27 19:27:42.873724 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.873710    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:27:42.874236 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.874214    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/0.log"
Jul 27 19:27:42.876195 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.876174    2579 generic.go:356] "Generic (PLEG): container finished" podID="68a01e15-aea1-4950-bd2a-47bac1e23587" containerID="51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf" exitCode=1
Jul 27 19:27:42.876335 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.876216    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerDied","Data":"51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf"}
Jul 27 19:27:42.876905 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.876886    2579 scope.go:117] "RemoveContainer" containerID="51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf"
Jul 27 19:27:42.877036 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.877018    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"ovnkube-controller\" with CrashLoopBackOff: \"back-off 10s restarting failed container=ovnkube-controller pod=ovnkube-node-rmqjz_openshift-ovn-kubernetes(68a01e15-aea1-4950-bd2a-47bac1e23587)\"" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" podUID="68a01e15-aea1-4950-bd2a-47bac1e23587"
Jul 27 19:27:42.883223 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:42.883209    2579 scope.go:117] "RemoveContainer" containerID="4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b"
Jul 27 19:27:42.884368 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.884310    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:56+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_1f8e83e1-1f00-4de3-bb21-f101d2dd49cb\\\\n2026-07-27T19:26:57+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_1f8e83e1-1f00-4de3-bb21-f101d2dd49cb to /host/opt/cni/bin/\\\\n2026-07-27T19:26:57Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:57Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:27:42Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:42.903442 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.903287    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"68a01e15-aea1-4950-bd2a-47bac1e23587\\\"},\\\"status\\\":{\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://29fb8aad8138d6751c5478b6e6ee863d490cd5d96c5daf833ffd482911702ca7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://bf555d82605764ea5d641a51b2077f8d38a5b541372a235c743895b1a2a676ec\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dc6db2c752d35e3d7a43c5ed94803dc3a3a192fda87035932d3dbbbdc8f9fda1\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://f934629c304dec56b6341a0d62b35f7d066b0c3c691189ce61cb632ff6355ccd\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://cf429e393aca3c0ea22a832074b0a382e4c67432b08543028e41e5f1063bf452\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dbf59404bbabd9ff4b2518939d7428eb39da70dbd6816197270bd53f3aad9861\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"},\\\"containerID\\\":\\\"cri-o://4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"message\\\":\\\"reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressservice/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:41.231603    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.EgressIP\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressip/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:41.233366    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1alpha1.AdminNetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"sigs.k8s.io/network-policy-api/pkg/client/informers/externalversions/factory.go:141\\\\\\\"\\\\nI0727 19:27:41.233418    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1alpha1.BaselineAdminNetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"sigs.k8s.io/network-policy-api/pkg/client/informers/externalversions/factory.go:141\\\\\\\"\\\\nI0727 19:27:41.236931    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Namespace\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:41.237381    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Service\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:41.237570    3403 factory.go:718] Stopping watch factory\\\\nI0727 19:27:41.237643    3403 ovnkube.go:653] Stopped ovnkube\\\\nI0727 19:27:41.237667    3403 server.go:251] Stopping metrics server at address \\\\\\\"127.0.0.1:29103\\\\\\\"\\\\nF0727 19:27:41.237717    3403 ovnkube.go:149] failed to run ovnkube: [failed to start network controller: failed to start default network controller - while waiting for \\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://a7d254f33d22b5634fd96a603f3656601baff0c9fcf80f72757088742b23cf0f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:13Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-rmqjz\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:27:42.917016 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.916902    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:42.928356 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.928290    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:56+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_1f8e83e1-1f00-4de3-bb21-f101d2dd49cb\\\\n2026-07-27T19:26:57+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_1f8e83e1-1f00-4de3-bb21-f101d2dd49cb to /host/opt/cni/bin/\\\\n2026-07-27T19:26:57Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:57Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:27:42Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:42.945153 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.944973    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"68a01e15-aea1-4950-bd2a-47bac1e23587\\\"},\\\"status\\\":{\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://29fb8aad8138d6751c5478b6e6ee863d490cd5d96c5daf833ffd482911702ca7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://bf555d82605764ea5d641a51b2077f8d38a5b541372a235c743895b1a2a676ec\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dc6db2c752d35e3d7a43c5ed94803dc3a3a192fda87035932d3dbbbdc8f9fda1\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://f934629c304dec56b6341a0d62b35f7d066b0c3c691189ce61cb632ff6355ccd\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://cf429e393aca3c0ea22a832074b0a382e4c67432b08543028e41e5f1063bf452\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dbf59404bbabd9ff4b2518939d7428eb39da70dbd6816197270bd53f3aad9861\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"},\\\"containerID\\\":\\\"cri-o://51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"message\\\":\\\"reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressservice/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:41.231603    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.EgressIP\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressip/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:41.233366    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1alpha1.AdminNetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"sigs.k8s.io/network-policy-api/pkg/client/informers/externalversions/factory.go:141\\\\\\\"\\\\nI0727 19:27:41.233418    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1alpha1.BaselineAdminNetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"sigs.k8s.io/network-policy-api/pkg/client/informers/externalversions/factory.go:141\\\\\\\"\\\\nI0727 19:27:41.236931    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Namespace\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:41.237381    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Service\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:41.237570    3403 factory.go:718] Stopping watch factory\\\\nI0727 19:27:41.237643    3403 ovnkube.go:653] Stopped ovnkube\\\\nI0727 19:27:41.237667    3403 server.go:251] Stopping metrics server at address \\\\\\\"127.0.0.1:29103\\\\\\\"\\\\nF0727 19:27:41.237717    3403 ovnkube.go:149] failed to run ovnkube: [failed to start network controller: failed to start default network controller - while waiting for \\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"1.EgressFirewall\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressfirewall/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:42.819021    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Pod\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.821661    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Namespace\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.821695    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Service\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823536    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.EndpointSlice\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823617    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Node\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823768    4083 factory.go:718] Stopping watch factory\\\\nI0727 19:27:42.823617    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.NetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.824007    4083 ovnkube.go:653] Stopped ovnkube\\\\nI0727 19:27:42.824038    4083 server.go:251] Stopping metrics server at address \\\\\\\"127.0.0.1:29103\\\\\\\"\\\\nF0727 19:27:42.824085    4083 ovnkube.go:149] failed to run ovnkube: [failed to start network controller: failed to start default network controller - while waiting for \\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:27:41Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://a7d254f33d22b5634fd96a603f3656601baff0c9fcf80f72757088742b23cf0f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:13Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-rmqjz\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:27:42.958750 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:42.958636    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:43.482681 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:43.482648    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:43.482862 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:43.482770    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:43.879783 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:43.879708    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:27:43.881416 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:43.881401    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:27:44.477759 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:44.477721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:44.483421 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:44.483399    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:44.483584 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:44.483506    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:44.483584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:44.483567    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:44.483700 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:44.483677    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:44.578039 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:44.578003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:44.678047 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:44.678012    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:45.483400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:45.483361    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:45.483755 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:45.483467    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:46.483200 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:46.483158    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:46.483393 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:46.483158    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:46.483393 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:46.483307    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:46.483393 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:46.483349    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:46.663035 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:46.662995    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:47.313625 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:47.313588    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:47.483082 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:47.483051    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:47.483210 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:47.483155    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:48.354470 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.354439    2579 kubelet_node_status.go:448] "Node not becoming ready in time after startup"
Jul 27 19:27:48.359554 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.359514    2579 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 27 19:27:48.483648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:48.483604    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:48.483788 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:48.483661    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:48.483788 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.483757    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:48.483873 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.483856    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:48.486299 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:48.486125    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:27:48.486986 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:48.486968    2579 scope.go:117] "RemoveContainer" containerID="51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf"
Jul 27 19:27:48.487150 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.487131    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"ovnkube-controller\" with CrashLoopBackOff: \"back-off 10s restarting failed container=ovnkube-controller pod=ovnkube-node-rmqjz_openshift-ovn-kubernetes(68a01e15-aea1-4950-bd2a-47bac1e23587)\"" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" podUID="68a01e15-aea1-4950-bd2a-47bac1e23587"
Jul 27 19:27:48.497195 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.497125    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:56+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_1f8e83e1-1f00-4de3-bb21-f101d2dd49cb\\\\n2026-07-27T19:26:57+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_1f8e83e1-1f00-4de3-bb21-f101d2dd49cb to /host/opt/cni/bin/\\\\n2026-07-27T19:26:57Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:57Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:27:42Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:48.516367 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.516167    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"68a01e15-aea1-4950-bd2a-47bac1e23587\\\"},\\\"status\\\":{\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://29fb8aad8138d6751c5478b6e6ee863d490cd5d96c5daf833ffd482911702ca7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://bf555d82605764ea5d641a51b2077f8d38a5b541372a235c743895b1a2a676ec\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dc6db2c752d35e3d7a43c5ed94803dc3a3a192fda87035932d3dbbbdc8f9fda1\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://f934629c304dec56b6341a0d62b35f7d066b0c3c691189ce61cb632ff6355ccd\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://cf429e393aca3c0ea22a832074b0a382e4c67432b08543028e41e5f1063bf452\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dbf59404bbabd9ff4b2518939d7428eb39da70dbd6816197270bd53f3aad9861\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"},\\\"containerID\\\":\\\"cri-o://51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4fc8a5a5a18d9b2e33755e46b7e2f589fec6d7a6a1b7d8b079948a58f4cdc32b\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:41Z\\\",\\\"message\\\":\\\"reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressservice/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:41.231603    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.EgressIP\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressip/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:41.233366    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1alpha1.AdminNetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"sigs.k8s.io/network-policy-api/pkg/client/informers/externalversions/factory.go:141\\\\\\\"\\\\nI0727 19:27:41.233418    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1alpha1.BaselineAdminNetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"sigs.k8s.io/network-policy-api/pkg/client/informers/externalversions/factory.go:141\\\\\\\"\\\\nI0727 19:27:41.236931    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Namespace\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:41.237381    3403 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Service\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:41.237570    3403 factory.go:718] Stopping watch factory\\\\nI0727 19:27:41.237643    3403 ovnkube.go:653] Stopped ovnkube\\\\nI0727 19:27:41.237667    3403 server.go:251] Stopping metrics server at address \\\\\\\"127.0.0.1:29103\\\\\\\"\\\\nF0727 19:27:41.237717    3403 ovnkube.go:149] failed to run ovnkube: [failed to start network controller: failed to start default network controller - while waiting for \\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"1.EgressFirewall\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressfirewall/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:42.819021    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Pod\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.821661    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Namespace\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.821695    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Service\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823536    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.EndpointSlice\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823617    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Node\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823768    4083 factory.go:718] Stopping watch factory\\\\nI0727 19:27:42.823617    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.NetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.824007    4083 ovnkube.go:653] Stopped ovnkube\\\\nI0727 19:27:42.824038    4083 server.go:251] Stopping metrics server at address \\\\\\\"127.0.0.1:29103\\\\\\\"\\\\nF0727 19:27:42.824085    4083 ovnkube.go:149] failed to run ovnkube: [failed to start network controller: failed to start default network controller - while waiting for \\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:27:41Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://a7d254f33d22b5634fd96a603f3656601baff0c9fcf80f72757088742b23cf0f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:13Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-rmqjz\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:27:48.532086 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.531969    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:48.544370 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.544289    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd1a649-a8e2-4657-ba0d-b19301043af8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"},\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://17c3316fa88dafb8ae5fc30c6e7b08c9e15d65c82eab5713ada349978a20573d\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:26:55Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:10+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e\\\\n2026-07-27T19:26:10+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_7a75d146-047b-49b7-87ed-ebbc045a2b0e to /host/opt/cni/bin/\\\\n2026-07-27T19:26:10Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:10Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:26:55Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"65Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"2026-07-27T19:26:56+00:00 [cnibincopy] Successfully copied files in /usr/src/multus-cni/rhel9/bin/ to /host/opt/cni/bin/upgrade_1f8e83e1-1f00-4de3-bb21-f101d2dd49cb\\\\n2026-07-27T19:26:57+00:00 [cnibincopy] Successfully moved files in /host/opt/cni/bin/upgrade_1f8e83e1-1f00-4de3-bb21-f101d2dd49cb to /host/opt/cni/bin/\\\\n2026-07-27T19:26:57Z [verbose] multus-daemon started\\\\n2026-07-27T19:26:57Z [verbose] Readiness Indicator file check\\\\n2026-07-27T19:27:42Z [error] have you checked that your default network is ready? still waiting for readinessindicatorfile @ /host/run/multus/cni/net.d/10-ovn-kubernetes.conf. pollimmediate error: timed out waiting for the condition\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:56Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bv7vb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-multus\"/\"multus-q2v4h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-q2v4h"
Jul 27 19:27:48.564033 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.563874    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"68a01e15-aea1-4950-bd2a-47bac1e23587\\\"},\\\"status\\\":{\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://29fb8aad8138d6751c5478b6e6ee863d490cd5d96c5daf833ffd482911702ca7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://bf555d82605764ea5d641a51b2077f8d38a5b541372a235c743895b1a2a676ec\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:373a63186d156b4ce0ae2d8560e108441d17b09be34e6d6db2b3486bc43f7dfd\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":65534,\\\"supplementalGroups\\\":[65534],\\\"uid\\\":65534}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dc6db2c752d35e3d7a43c5ed94803dc3a3a192fda87035932d3dbbbdc8f9fda1\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://f934629c304dec56b6341a0d62b35f7d066b0c3c691189ce61cb632ff6355ccd\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:11Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"},\\\"containerID\\\":\\\"cri-o://cf429e393aca3c0ea22a832074b0a382e4c67432b08543028e41e5f1063bf452\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"20Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://dbf59404bbabd9ff4b2518939d7428eb39da70dbd6816197270bd53f3aad9861\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"},\\\"containerID\\\":\\\"cri-o://51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-27T19:27:42Z\\\",\\\"message\\\":\\\"1.EgressFirewall\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"github.com/openshift/ovn-kubernetes/go-controller/pkg/crd/egressfirewall/v1/apis/informers/externalversions/factory.go:129\\\\\\\"\\\\nI0727 19:27:42.819021    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Pod\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.821661    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Namespace\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.821695    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Service\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823536    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.EndpointSlice\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823617    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.Node\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.823768    4083 factory.go:718] Stopping watch factory\\\\nI0727 19:27:42.823617    4083 reflector.go:373] \\\\\\\"Stopping reflector\\\\\\\" type=\\\\\\\"*v1.NetworkPolicy\\\\\\\" resyncPeriod=\\\\\\\"0s\\\\\\\" reflector=\\\\\\\"k8s.io/client-go/informers/factory.go:161\\\\\\\"\\\\nI0727 19:27:42.824007    4083 ovnkube.go:653] Stopped ovnkube\\\\nI0727 19:27:42.824038    4083 server.go:251] Stopping metrics server at address \\\\\\\"127.0.0.1:29103\\\\\\\"\\\\nF0727 19:27:42.824085    4083 ovnkube.go:149] failed to run ovnkube: [failed to start network controller: failed to start default network controller - while waiting for \\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-27T19:27:41Z\\\"}},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"600Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"message\\\":\\\"back-off 10s restarting failed container=ovnkube-controller pod=ovnkube-node-rmqjz_openshift-ovn-kubernetes(68a01e15-aea1-4950-bd2a-47bac1e23587)\\\",\\\"reason\\\":\\\"CrashLoopBackOff\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"containerID\\\":\\\"cri-o://a7d254f33d22b5634fd96a603f3656601baff0c9fcf80f72757088742b23cf0f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16ca8b7c1184a105aac97293a35c94373262103579713746b69f4f125185b301\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"300Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:13Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-97hpq\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-rmqjz\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:27:48.579398 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:48.579277    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"56643b2e-0ea2-498b-aed2-e185018e7028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-27T19:26:27Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://bafff8a1cc1220c5d8216e3b38b5b725c46ebb4e363a921c2f4e1f9d62654445\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea7622cd6145f37fcac7b916bd4464831e2efdf19ebe0ec41c27047c814f23f0\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:09a988dc79ac3bbc5aff0a5cae44cb8ae06b7d21a0c28d30499f6b5f63fcefd8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-27T19:26:26Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://648a52b8af4b701b33d5c06e66204de5e8166ad44da8a68b66a1488e90010e42\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:10Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:10Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c672072cc02663bcd473a2edc6c5142d36ddb28dea685e0cfc5d06f52e03d64\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e31a4f12fe8fbb2afafbc3ba872caeb4ad868cb0463c49013a083052f459af5f\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15259073275e2789accc887bd84daeb72a005385030ffcfd81af0ece73a97191\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://bbbd0eea98ea04abbfacd10ebe76761a97a00a86befbb1334302307d0ee8211c\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:18Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:18Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:64a796986118e9bcb9f32dd15fba29e2aa5906f3068c8209a3ad21b66cd7fe06\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:062f884217d33f8982f101f5b2bee51590a578164f2707db799b0dce80c399e8\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4c9faad8ef098d513c2efe2c27e26d34fa868ce5332d2226d8a86a04e2ab1d00\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:20Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:19Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://de264dcf4c7bda3226589a9d36d64a5b4d2b4cae89a53d1e16c26bb512a2e3df\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:24Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:24Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e951c133c66e9b456835de5087842d6fc86fecfe5aad3ca560fdd9c719ed8ba\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f42f4c14e88b49108b37c533fbbc282996a50a648f57db40690d2b0e8d5c811\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://d2d68f441191e26fa05a89e4f84161dc081b0899e36ef4c179d37df2832db129\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-07-27T19:26:25Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-07-27T19:26:25Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/etc/whereabouts/config\\\",\\\"name\\\":\\\"whereabouts-flatfile-configmap\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-hqzjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Running\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-hcqwt\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-98c63052-5512-48ed-9e58-8008512a0fb1.svc:9743/pod?timeout=10s\": dial tcp 172.30.119.215:9743: connect: connection refused" pod="openshift-multus/multus-additional-cni-plugins-hcqwt"
Jul 27 19:27:48.957168 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:48.957126    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:49.055589 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:49.055546    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:49.155055 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:49.155024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:49.217169 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:49.217046    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:49.483418 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:49.483340    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:49.483752 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:49.483446    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:49.634018 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:49.633984    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:49.695575 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:49.695551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:49.759478 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:49.759395    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:50.055853 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:50.055765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:50.119562 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:50.119520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:50.484060 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:50.484017    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:50.484442 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:50.484147    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:50.484442 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:50.484150    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:50.484570 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:50.484547    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:50.551210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:50.551177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:50.614984 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:50.614944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:51.149167 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:51.149124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:51.211984 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:51.211947    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:51.274809 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:51.274781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:51.337710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:51.337677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:51.482778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:51.482747    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:51.482940 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:51.482870    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:52.038191 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:52.038155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:52.103065 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:52.103033    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:52.432157 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:52.432124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:52.483305 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:52.483280    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:52.483382 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:52.483314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:52.483424 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:52.483403    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:52.483537 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:52.483513    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:52.484997 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:52.484967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:52.857211 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:52.857134    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:52.917163 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:52.917095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:53.225410 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:53.225379    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:53.326580 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:53.326545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:53.360996 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:53.360960    2579 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 27 19:27:53.483021 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:53.482934    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:53.483183 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:53.483052    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:53.524658 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:53.524632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:53.690127 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:53.690092    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:53.778367 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:53.778288    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:53.838850 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:53.838813    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:54.092549 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.092470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:54.151960 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.151929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:54.441830 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.441790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:54.482829 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.482802    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:54.482936 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:54.482920    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:54.483020 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.483000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:54.483132 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:54.483113    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:54.507448 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.507418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:54.611260 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.611215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:54.673932 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.673901    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:54.839301 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.839197    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:54.902840 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:54.902808    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:55.482639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:55.482601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:55.483032 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:55.482700    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:55.635342 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:55.635309    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:55.736703 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:55.736628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.038127 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.038051    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.136613 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.136574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.236988 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.236954    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.482624 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.482591    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:56.482709 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.482595    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:56.483069 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.482937    2579 scope.go:117] "RemoveContainer" containerID="507471f98172675e025ee0cc0a3021a60bd11bbea6a31bb586a09bdd980fe224"
Jul 27 19:27:56.483069 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:56.482943    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:56.483069 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:56.483018    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:56.552917 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.552861    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hcqwt" podStartSLOduration=98.014831666 podStartE2EDuration="2m7.552845941s" totalImagesPullingTime="29.538014275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:56.552449848 +0000 UTC m=+128.643801345" watchObservedRunningTime="2026-07-27 19:27:56.552845941 +0000 UTC m=+128.644197439"
Jul 27 19:27:56.836179 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.836080    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.902322 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.902284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:56.910353 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.910334    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:27:56.910494 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:56.910402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2v4h" event={"ID":"bcd1a649-a8e2-4657-ba0d-b19301043af8","Type":"ContainerStarted","Data":"56cb4216cf6dae069c16dea5ccf9fe1b9b165123a327b24e0bb75642d5a86e71"}
Jul 27 19:27:57.268022 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:57.267989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:57.323629 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:57.323599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:57.385277 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:57.385227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:57.482475 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:57.482439    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:57.482625 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:57.482541    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:57.636636 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:57.636553    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:57.733743 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:57.733705    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:58.234771 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.234741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:58.236704 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.236685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:58.236771 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.236721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:58.236835 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.236823    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:58.236880 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.236871    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs podName:1681bd39-34f5-4469-8ed5-fc75330bad04 nodeName:}" failed. No retries permitted until 2026-07-27 19:30:00.236859172 +0000 UTC m=+252.328210647 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs") pod "network-metrics-daemon-qc4zf" (UID: "1681bd39-34f5-4469-8ed5-fc75330bad04") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:58.236928 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.236823    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:58.236968 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.236951    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret podName:185be14c-e7d0-494b-b626-94f381a9ab93 nodeName:}" failed. No retries permitted until 2026-07-27 19:30:00.236937702 +0000 UTC m=+252.328289178 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret") pod "global-pull-secret-syncer-wtrmw" (UID: "185be14c-e7d0-494b-b626-94f381a9ab93") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:58.297355 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.297323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:58.337440 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.337402    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:58.337602 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.337525    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:58.337602 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.337540    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:58.337602 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.337549    2579 projected.go:196] Error preparing data for projected volume kube-api-access-zzf8n for pod openshift-network-diagnostics/network-check-target-9km82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:58.337602 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.337600    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n podName:9f4eccf9-761d-4834-8f82-d244d41d38c7 nodeName:}" failed. No retries permitted until 2026-07-27 19:30:00.33758605 +0000 UTC m=+252.428937526 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf8n" (UniqueName: "kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n") pod "network-check-target-9km82" (UID: "9f4eccf9-761d-4834-8f82-d244d41d38c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:58.361357 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.361322    2579 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 27 19:27:58.485291 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.485187    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:27:58.485438 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.485318    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:27:58.485438 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.485376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:27:58.485504 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:58.485455    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:27:58.599722 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.599685    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:58.654527 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.654486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:58.709958 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:58.709926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:59.046347 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:59.046317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:59.121873 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:59.121841    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:27:59.482507 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:59.482468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:27:59.482657 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:27:59.482580    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:27:59.641774 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:27:59.641748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:00.044820 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:00.044788    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:00.144795 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:00.144763    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:00.483430 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:00.483396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:28:00.483594 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:00.483396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:28:00.483594 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:00.483521    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:28:00.483594 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:00.483573    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:28:01.048084 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:01.048052    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:01.238148 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:01.238104    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:01.302776 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:01.302693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:01.482624 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:01.482593    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:28:01.482811 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:01.482700    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:28:02.059352 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.059322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:02.439674 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.439632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:02.482628 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.482604    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:28:02.482761 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.482606    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:28:02.482761 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:02.482709    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:28:02.482845 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:02.482824    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:28:02.483423 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.483409    2579 scope.go:117] "RemoveContainer" containerID="51b6c28b182756571b8bec030b028c544df441b3f74ead8762e9876996b58fbf"
Jul 27 19:28:02.893663 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.893572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:02.929634 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.925610    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:28:02.929634 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.927969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz" event={"ID":"68a01e15-aea1-4950-bd2a-47bac1e23587","Type":"ContainerStarted","Data":"d89ead7fd372eb50957ce4c2afbac5119040f404e1bae870a9a5b18a871e5f09"}
Jul 27 19:28:02.929634 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.929448    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:28:02.956782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:02.956751    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:03.283159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.283119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:03.362049 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:03.361994    2579 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 27 19:28:03.483054 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.482787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:28:03.483197 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:03.483163    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:28:03.608503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.608355    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wtrmw"]
Jul 27 19:28:03.608742 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.608578    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:28:03.608742 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:03.608694    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:28:03.611633 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.611607    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qc4zf"]
Jul 27 19:28:03.611769 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.611727    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:28:03.611844 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:03.611822    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:28:03.612427 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.612365    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9km82"]
Jul 27 19:28:03.680936 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.680905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:03.738023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.737989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:03.838780 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.838746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:03.930216 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:03.930189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:28:03.930441 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:03.930294    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:28:04.262050 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:04.261972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:04.346159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:04.346120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:04.437509 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:04.437479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:04.498795 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:04.498761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:04.559977 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:04.559892    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:04.777438 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:04.777400    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:04.878074 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:04.877990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:04.977994 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:04.977962    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:05.243371 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:05.243335    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:05.483177 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:05.483143    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:28:05.483177 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:05.483161    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:28:05.483707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:05.483269    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:28:05.483707 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:05.483287    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:28:05.483707 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:05.483398    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:28:05.483707 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:05.483476    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:28:05.608477 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:05.608364    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:06.162330 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:06.162283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:06.228466 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:06.228434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:06.952479 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:06.952438    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:07.022752 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:07.022715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:07.122473 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:07.122439    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:07.222961 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:07.222859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:07.482645 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:07.482558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:28:07.482811 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:07.482558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:28:07.482811 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:07.482676    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qc4zf" podUID="1681bd39-34f5-4469-8ed5-fc75330bad04"
Jul 27 19:28:07.482811 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:07.482558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:28:07.482811 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:07.482746    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wtrmw" podUID="185be14c-e7d0-494b-b626-94f381a9ab93"
Jul 27 19:28:07.483011 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:07.482838    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9km82" podUID="9f4eccf9-761d-4834-8f82-d244d41d38c7"
Jul 27 19:28:07.753913 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:07.753826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:09.167579 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.167533    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:09.483192 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.483161    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:28:09.483360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.483173    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:28:09.483360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.483173    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:28:09.487218 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.487199    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4682l\""
Jul 27 19:28:09.487368 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.487222    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 19:28:09.487368 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.487225    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 19:28:09.487368 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.487224    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 19:28:09.487368 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.487199    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xt9mb\""
Jul 27 19:28:09.487368 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.487267    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 19:28:09.920039 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:09.919958    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vzl59_3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9/dns-node-resolver/0.log"
Jul 27 19:28:10.671428 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:10.671392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:10.751297 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:10.751265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:10.919879 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:10.919852    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-spvgm_9138d8cc-fdd4-4883-84be-0600b09f69a8/node-ca/0.log"
Jul 27 19:28:11.658042 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:11.658014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:11.759478 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:11.759437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:11.894814 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:11.894775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:12.002283 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:12.002225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:12.102123 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:12.102081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:12.203255 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:12.203206    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:12.365828 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:12.365745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:12.431406 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:12.431382    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-59.ec2.internal" event="NodeReady"
Jul 27 19:28:13.153856 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:13.153823    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:13.280170 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:13.280137    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:13.561284 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:13.561230    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:13.751472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:13.751439    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:14.320125 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:14.320098    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/egress-router-binary-copy/0.log"
Jul 27 19:28:14.365716 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:14.365677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:14.520370 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:14.520347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/cni-plugins/0.log"
Jul 27 19:28:14.719596 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:14.719574    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/bond-cni-plugin/0.log"
Jul 27 19:28:14.773436 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:14.773399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:14.921570 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:14.921543    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/routeoverride-cni/0.log"
Jul 27 19:28:15.120399 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:15.120332    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/whereabouts-cni-bincopy/0.log"
Jul 27 19:28:15.162536 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:15.162505    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:15.319617 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:15.319589    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/whereabouts-cni/0.log"
Jul 27 19:28:15.361657 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:15.361628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:15.520516 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:15.520492    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/kube-multus-additional-cni-plugins/0.log"
Jul 27 19:28:15.569227 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:15.569179    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:15.730009 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:15.729978    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:15.982026 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:15.981996    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:16.360217 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:16.360129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:16.561931 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:16.561899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:17.198965 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:17.198929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:17.320122 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:17.320099    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:28:17.360774 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:17.360735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:17.520763 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:17.520690    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/2.log"
Jul 27 19:28:17.556936 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:17.556899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:18.075522 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:18.075489    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:18.562792 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:18.562756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:19.303407 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:19.303368    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:19.389546 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:19.389512    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:19.552766 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:19.552734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:19.694285 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:19.694256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:20.758979 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:20.758949    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:20.960693 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:20.960660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:21.182658 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:21.182622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:21.368204 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:21.368171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:21.592617 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:21.592525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:22.155591 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:22.155553    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:22.365719 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:22.365678    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:23.522702 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:23.522667    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:28:23.722152 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:23.722121    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovn-controller/0.log"
Jul 27 19:28:23.922422 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:23.922339    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovn-acl-logging/0.log"
Jul 27 19:28:24.122798 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:24.122762    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/kube-rbac-proxy-node/0.log"
Jul 27 19:28:24.322896 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:24.322866    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/kube-rbac-proxy-ovn-metrics/0.log"
Jul 27 19:28:24.521148 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:24.521123    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/northd/0.log"
Jul 27 19:28:24.721631 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:24.721607    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/nbdb/0.log"
Jul 27 19:28:24.920751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:24.920720    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/sbdb/0.log"
Jul 27 19:28:25.122488 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:25.122413    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/2.log"
Jul 27 19:28:25.649020 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:25.648989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:25.728461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:25.728423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:27.920057 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:27.920030    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-cscws_09e9d3cf-99cc-4c17-8b64-1db7729420e7/iptables-alerter/0.log"
Jul 27 19:28:28.232508 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:28.232466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:28.333166 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:28.333132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:28.433729 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:28.433696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:28.538679 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:28.538596    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:29.036757 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:29.036722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:29.140630 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:29.140595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:29.312972 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:29.312891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:29.395961 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:29.395925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:29.511027 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:29.510986    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:30.116847 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.116818    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:30.320406 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.320368    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:30.467260 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.467204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:30.891106 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.891014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:30.958238 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.958202    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r872m"]
Jul 27 19:28:30.960462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.960444    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qd2hp"]
Jul 27 19:28:30.960593 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.960577    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:30.962408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.962389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:30.963213 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.963187    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 19:28:30.963213 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.963198    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 19:28:30.963360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.963274    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qd54j\""
Jul 27 19:28:30.963360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.963280    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 19:28:30.963360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.963321    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 19:28:30.964900 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.964874    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 19:28:30.965042 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.964923    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 19:28:30.965042 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.965008    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ctr9s\""
Jul 27 19:28:30.966563 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.966539    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:30.973569 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.973550    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qd2hp"]
Jul 27 19:28:30.974676 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:30.974656    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r872m"]
Jul 27 19:28:31.059461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.059422    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s2rbr"]
Jul 27 19:28:31.061727 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.061708    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s2rbr"
Jul 27 19:28:31.064172 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.064151    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 19:28:31.064172 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.064165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nmpmh\""
Jul 27 19:28:31.064369 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.064178    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 19:28:31.064538 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.064523    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 19:28:31.072183 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.072163    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s2rbr"]
Jul 27 19:28:31.075681 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.075661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7f6643-4911-4092-9a15-38019dc60868-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.075790 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.075691    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrxh2\" (UniqueName: \"kubernetes.io/projected/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-kube-api-access-lrxh2\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.075790 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.075716    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-tmp-dir\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.075790 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.075736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7f6643-4911-4092-9a15-38019dc60868-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.075956 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.075852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85zhl\" (UniqueName: \"kubernetes.io/projected/4d7f6643-4911-4092-9a15-38019dc60868-kube-api-access-85zhl\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.075956 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.075906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-config-volume\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.075956 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.075932    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-metrics-tls\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.076089 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.075973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d7f6643-4911-4092-9a15-38019dc60868-crio-socket\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.076089 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.076000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d7f6643-4911-4092-9a15-38019dc60868-data-volume\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.111987 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.111947    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:31.176807 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.176734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-tmp-dir\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.176807 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.176773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7f6643-4911-4092-9a15-38019dc60868-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.176834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85zhl\" (UniqueName: \"kubernetes.io/projected/4d7f6643-4911-4092-9a15-38019dc60868-kube-api-access-85zhl\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.176875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-config-volume\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.176899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-metrics-tls\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.176937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d7f6643-4911-4092-9a15-38019dc60868-crio-socket\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.176959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d7f6643-4911-4092-9a15-38019dc60868-data-volume\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.176983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7f6643-4911-4092-9a15-38019dc60868-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.177012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23-cert\") pod \"ingress-canary-s2rbr\" (UID: \"c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23\") " pod="openshift-ingress-canary/ingress-canary-s2rbr"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.177039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgdwv\" (UniqueName: \"kubernetes.io/projected/c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23-kube-api-access-zgdwv\") pod \"ingress-canary-s2rbr\" (UID: \"c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23\") " pod="openshift-ingress-canary/ingress-canary-s2rbr"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.177049    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d7f6643-4911-4092-9a15-38019dc60868-crio-socket\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.177064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrxh2\" (UniqueName: \"kubernetes.io/projected/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-kube-api-access-lrxh2\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.177319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.177149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-tmp-dir\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.177764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.177369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d7f6643-4911-4092-9a15-38019dc60868-data-volume\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.177764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.177557    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-config-volume\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.177831 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.177801    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7f6643-4911-4092-9a15-38019dc60868-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.180680 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.180664    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-metrics-tls\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.180751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.180664    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7f6643-4911-4092-9a15-38019dc60868-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.186148 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.186125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrxh2\" (UniqueName: \"kubernetes.io/projected/3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a-kube-api-access-lrxh2\") pod \"dns-default-qd2hp\" (UID: \"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a\") " pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.187091 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.187069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85zhl\" (UniqueName: \"kubernetes.io/projected/4d7f6643-4911-4092-9a15-38019dc60868-kube-api-access-85zhl\") pod \"insights-runtime-extractor-r872m\" (UID: \"4d7f6643-4911-4092-9a15-38019dc60868\") " pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.212446 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.212402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:31.272453 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.272423    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r872m"
Jul 27 19:28:31.278099 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.278078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:31.278298 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.278270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23-cert\") pod \"ingress-canary-s2rbr\" (UID: \"c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23\") " pod="openshift-ingress-canary/ingress-canary-s2rbr"
Jul 27 19:28:31.278405 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.278314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgdwv\" (UniqueName: \"kubernetes.io/projected/c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23-kube-api-access-zgdwv\") pod \"ingress-canary-s2rbr\" (UID: \"c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23\") " pod="openshift-ingress-canary/ingress-canary-s2rbr"
Jul 27 19:28:31.280648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.280626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23-cert\") pod \"ingress-canary-s2rbr\" (UID: \"c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23\") " pod="openshift-ingress-canary/ingress-canary-s2rbr"
Jul 27 19:28:31.289558 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.289527    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgdwv\" (UniqueName: \"kubernetes.io/projected/c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23-kube-api-access-zgdwv\") pod \"ingress-canary-s2rbr\" (UID: \"c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23\") " pod="openshift-ingress-canary/ingress-canary-s2rbr"
Jul 27 19:28:31.329096 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.329055    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:31.370775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.370748    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s2rbr"
Jul 27 19:28:31.416878 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.416786    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r872m"]
Jul 27 19:28:31.432140 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.432042    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qd2hp"]
Jul 27 19:28:31.435571 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.434655    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:31.436775 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:31.436714    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3737a0cf_1d14_4e8e_9f28_dc12fd2ff68a.slice/crio-fe835801c2fe2577e31a1798acd967f85d512dedfba50afa8d78b492308e666e WatchSource:0}: Error finding container fe835801c2fe2577e31a1798acd967f85d512dedfba50afa8d78b492308e666e: Status 404 returned error can't find the container with id fe835801c2fe2577e31a1798acd967f85d512dedfba50afa8d78b492308e666e
Jul 27 19:28:31.500931 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.500876    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s2rbr"]
Jul 27 19:28:31.505509 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:31.505482    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c1a9a5_7b01_4855_a9f8_809dbf3c9e23.slice/crio-f7aa100b43d793aa10b2ccc7cdd1b6c6d32b36504b2b584e87841638660e7944 WatchSource:0}: Error finding container f7aa100b43d793aa10b2ccc7cdd1b6c6d32b36504b2b584e87841638660e7944: Status 404 returned error can't find the container with id f7aa100b43d793aa10b2ccc7cdd1b6c6d32b36504b2b584e87841638660e7944
Jul 27 19:28:31.994589 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.994547    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qd2hp" event={"ID":"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a","Type":"ContainerStarted","Data":"fe835801c2fe2577e31a1798acd967f85d512dedfba50afa8d78b492308e666e"}
Jul 27 19:28:31.996285 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:31.996187    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s2rbr" event={"ID":"c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23","Type":"ContainerStarted","Data":"f7aa100b43d793aa10b2ccc7cdd1b6c6d32b36504b2b584e87841638660e7944"}
Jul 27 19:28:32.001621 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:32.001540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r872m" event={"ID":"4d7f6643-4911-4092-9a15-38019dc60868","Type":"ContainerStarted","Data":"9efd3876b27c169e28e8847a629bee695e56c6f27e4278990d608923ca23a748"}
Jul 27 19:28:33.942839 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:33.942809    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rmqjz"
Jul 27 19:28:34.060553 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:34.060490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:34.118017 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:34.117979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:34.185829 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:34.185793    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:37.018940 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:37.018862    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qd2hp" event={"ID":"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a","Type":"ContainerStarted","Data":"9cebcc2b44d5c234821559cc0f88ea07b8d7dcc891dff855222925e08c905872"}
Jul 27 19:28:37.022227 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:37.022144    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s2rbr" event={"ID":"c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23","Type":"ContainerStarted","Data":"e3a9c273e57a2b423009ea581cd1dd38567be48813b3f084ff60be20fed8213e"}
Jul 27 19:28:37.023920 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:37.023896    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r872m" event={"ID":"4d7f6643-4911-4092-9a15-38019dc60868","Type":"ContainerStarted","Data":"85a54d002a967e039982c901071a92b8b1aa292830f5631f4c85284430a8af5f"}
Jul 27 19:28:38.030050 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:38.030010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qd2hp" event={"ID":"3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a","Type":"ContainerStarted","Data":"452cc8e540b1e27f16e4df14a03445622b16c6d2fcf57ce4a47954b99abc4bf7"}
Jul 27 19:28:38.048608 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:38.048549    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s2rbr" podStartSLOduration=1.703808361 podStartE2EDuration="7.048527264s" totalImagesPullingTime="5.344718903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:37.037381198 +0000 UTC m=+169.128732698" watchObservedRunningTime="2026-07-27 19:28:38.048527264 +0000 UTC m=+170.139878761"
Jul 27 19:28:38.048963 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:38.048935    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qd2hp" podStartSLOduration=2.63659941 podStartE2EDuration="8.048925666s" totalImagesPullingTime="5.412326256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:38.047874988 +0000 UTC m=+170.139226511" watchObservedRunningTime="2026-07-27 19:28:38.048925666 +0000 UTC m=+170.140277147"
Jul 27 19:28:39.033708 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:39.033675    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:41.042400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:41.042363    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r872m" event={"ID":"4d7f6643-4911-4092-9a15-38019dc60868","Type":"ContainerStarted","Data":"c7bf43b39d3fe7bc1af8c443a3c002bdc4fde43466107a1d1491fd1ad4e28592"}
Jul 27 19:28:44.054529 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:44.054428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r872m" event={"ID":"4d7f6643-4911-4092-9a15-38019dc60868","Type":"ContainerStarted","Data":"07eec7848ca6fb0c62aa65c83acb87a2d767138dfb0f1ab60e9f8a83ec6d1b25"}
Jul 27 19:28:44.079681 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:44.079629    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r872m" podStartSLOduration=1.883698664 podStartE2EDuration="14.079613092s" totalImagesPullingTime="12.195914428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:28:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:44.078817698 +0000 UTC m=+176.170169195" watchObservedRunningTime="2026-07-27 19:28:44.079613092 +0000 UTC m=+176.170964590"
Jul 27 19:28:44.944461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:44.944426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:45.009884 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:45.009837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:46.039098 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:46.039065    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qd2hp"
Jul 27 19:28:48.097115 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.097063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:48.263439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.263408    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"]
Jul 27 19:28:48.266264 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.266230    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"
Jul 27 19:28:48.268884 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.268859    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 27 19:28:48.269013 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.268997    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-r9dd4\""
Jul 27 19:28:48.275049 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.275025    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"]
Jul 27 19:28:48.304888 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.304858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ba4786db-13fb-41ce-bfe3-d564d2ab98c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8rf2s\" (UID: \"ba4786db-13fb-41ce-bfe3-d564d2ab98c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"
Jul 27 19:28:48.349318 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.349228    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-568774876c-4f2cf"]
Jul 27 19:28:48.352120 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.352100    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.354638 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.354612    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 27 19:28:48.354832 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.354817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 27 19:28:48.354890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.354817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gwndm\""
Jul 27 19:28:48.355094 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.355068    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 27 19:28:48.355094 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.355086    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 27 19:28:48.355275 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.355111    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 27 19:28:48.355275 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.355129    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 27 19:28:48.355275 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.355127    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 27 19:28:48.371031 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.371012    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568774876c-4f2cf"]
Jul 27 19:28:48.405417 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.405387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ba4786db-13fb-41ce-bfe3-d564d2ab98c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8rf2s\" (UID: \"ba4786db-13fb-41ce-bfe3-d564d2ab98c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"
Jul 27 19:28:48.405417 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.405416    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-service-ca\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.405614 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.405435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-oauth-config\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.405614 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.405450    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-config\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.405614 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.405467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66gtl\" (UniqueName: \"kubernetes.io/projected/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-kube-api-access-66gtl\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.405614 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.405540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-oauth-serving-cert\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.405614 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.405589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-serving-cert\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.407680 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.407664    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ba4786db-13fb-41ce-bfe3-d564d2ab98c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8rf2s\" (UID: \"ba4786db-13fb-41ce-bfe3-d564d2ab98c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"
Jul 27 19:28:48.409214 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.409190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:48.506615 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.506578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-serving-cert\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.506785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.506654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-service-ca\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.506785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.506674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-oauth-config\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.506785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.506688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-config\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.506785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.506705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66gtl\" (UniqueName: \"kubernetes.io/projected/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-kube-api-access-66gtl\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.506981 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.506860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-oauth-serving-cert\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.507456 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.507436    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-config\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.507561 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.507460    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-oauth-serving-cert\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.507561 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.507536    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-service-ca\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.509201 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.509157    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-serving-cert\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.509201 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.509186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-oauth-config\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.509398 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.509299    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:48.514518 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.514500    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66gtl\" (UniqueName: \"kubernetes.io/projected/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-kube-api-access-66gtl\") pod \"console-568774876c-4f2cf\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") " pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.575490 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.575458    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"
Jul 27 19:28:48.661266 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.661215    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:48.699255 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.699212    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"]
Jul 27 19:28:48.702640 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:48.702611    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4786db_13fb_41ce_bfe3_d564d2ab98c9.slice/crio-ec894843967c7d1acf1fa1ddde61323b9dcfb2508a45f4b33b1b43c6943c0dec WatchSource:0}: Error finding container ec894843967c7d1acf1fa1ddde61323b9dcfb2508a45f4b33b1b43c6943c0dec: Status 404 returned error can't find the container with id ec894843967c7d1acf1fa1ddde61323b9dcfb2508a45f4b33b1b43c6943c0dec
Jul 27 19:28:48.723129 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.723095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:48.811667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.811636    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568774876c-4f2cf"]
Jul 27 19:28:48.814900 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:48.814871    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f81b5b_648f_49ea_ae0f_fe3bd82e3f27.slice/crio-ddcbbc8b7d57a1a67f1b0a02aabc2196bfb32e31400002aa41050168d643a16d WatchSource:0}: Error finding container ddcbbc8b7d57a1a67f1b0a02aabc2196bfb32e31400002aa41050168d643a16d: Status 404 returned error can't find the container with id ddcbbc8b7d57a1a67f1b0a02aabc2196bfb32e31400002aa41050168d643a16d
Jul 27 19:28:48.846739 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.846699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:48.976584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:48.976559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:49.053500 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.053464    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:49.069893 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.069864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s" event={"ID":"ba4786db-13fb-41ce-bfe3-d564d2ab98c9","Type":"ContainerStarted","Data":"ec894843967c7d1acf1fa1ddde61323b9dcfb2508a45f4b33b1b43c6943c0dec"}
Jul 27 19:28:49.070940 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.070911    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568774876c-4f2cf" event={"ID":"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27","Type":"ContainerStarted","Data":"ddcbbc8b7d57a1a67f1b0a02aabc2196bfb32e31400002aa41050168d643a16d"}
Jul 27 19:28:49.732783 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.732748    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5965b89948-nqdwb"]
Jul 27 19:28:49.736167 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.735830    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.741106 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.741074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:49.754561 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.754347    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 27 19:28:49.754971 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.754880    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5965b89948-nqdwb"]
Jul 27 19:28:49.810549 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.810508    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:49.816722 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.816652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-oauth-config\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.816722 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.816700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-config\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.816942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.816738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-service-ca\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.816942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.816762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqkh\" (UniqueName: \"kubernetes.io/projected/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-kube-api-access-pfqkh\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.816942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.816789    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-serving-cert\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.816942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.816816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-trusted-ca-bundle\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.816942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.816842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-oauth-serving-cert\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.918408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.917594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-oauth-config\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.918408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.917641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-config\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.918408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.917669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-service-ca\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.918408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.917695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqkh\" (UniqueName: \"kubernetes.io/projected/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-kube-api-access-pfqkh\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.918408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.917724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-serving-cert\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.918408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.917750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-trusted-ca-bundle\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.918408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.917777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-oauth-serving-cert\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.918911 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.918843    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-service-ca\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.919112 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.919086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-config\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.919227 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.919205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-oauth-serving-cert\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.919991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.919944    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-trusted-ca-bundle\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.922607 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.922584    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-serving-cert\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.923638 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.923613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-oauth-config\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:49.929119 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:49.929073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqkh\" (UniqueName: \"kubernetes.io/projected/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-kube-api-access-pfqkh\") pod \"console-5965b89948-nqdwb\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") " pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:50.054518 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:50.054430    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:28:50.402714 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:50.402683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5965b89948-nqdwb"]
Jul 27 19:28:50.405725 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:50.405697    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfba2a95_8524_4c55_bc1f_9c7cc3abc933.slice/crio-be5d0a7ddeafb066576c96942539bb44724ebe323d314096652f3cf8f2095cce WatchSource:0}: Error finding container be5d0a7ddeafb066576c96942539bb44724ebe323d314096652f3cf8f2095cce: Status 404 returned error can't find the container with id be5d0a7ddeafb066576c96942539bb44724ebe323d314096652f3cf8f2095cce
Jul 27 19:28:51.079743 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.079695    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s" event={"ID":"ba4786db-13fb-41ce-bfe3-d564d2ab98c9","Type":"ContainerStarted","Data":"76ee1eca7c9390b9403014102ec35600a549f37009e306400d598e16db8253e0"}
Jul 27 19:28:51.080169 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.080061    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"
Jul 27 19:28:51.081134 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.081104    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5965b89948-nqdwb" event={"ID":"cfba2a95-8524-4c55-bc1f-9c7cc3abc933","Type":"ContainerStarted","Data":"be5d0a7ddeafb066576c96942539bb44724ebe323d314096652f3cf8f2095cce"}
Jul 27 19:28:51.085861 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.085837    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s"
Jul 27 19:28:51.098426 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.098373    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8rf2s" podStartSLOduration=1.486259086 podStartE2EDuration="3.098353636s" totalImagesPullingTime="1.61209455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:51.097546022 +0000 UTC m=+183.188897522" watchObservedRunningTime="2026-07-27 19:28:51.098353636 +0000 UTC m=+183.189705152"
Jul 27 19:28:51.167171 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.167096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:51.236440 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.236402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:51.303071 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.302987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:51.328555 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.328526    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qgb82"]
Jul 27 19:28:51.331887 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.331828    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.334286 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.334228    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 27 19:28:51.334935 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.334908    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 19:28:51.335163 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.335142    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 19:28:51.335967 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.335933    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 27 19:28:51.335967 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.335947    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dhl9b\""
Jul 27 19:28:51.336149 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.335927    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 19:28:51.340819 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.340795    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qgb82"]
Jul 27 19:28:51.377837 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.377803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:51.431135 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.431097    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cnzr\" (UniqueName: \"kubernetes.io/projected/dbaaea17-1a2c-406a-a748-03b901948ead-kube-api-access-4cnzr\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.431348 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.431207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbaaea17-1a2c-406a-a748-03b901948ead-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.431348 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.431260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbaaea17-1a2c-406a-a748-03b901948ead-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.431348 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.431310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dbaaea17-1a2c-406a-a748-03b901948ead-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.444037 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.443989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:51.532746 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.532708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbaaea17-1a2c-406a-a748-03b901948ead-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.532746 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.532750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbaaea17-1a2c-406a-a748-03b901948ead-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.532996 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.532788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dbaaea17-1a2c-406a-a748-03b901948ead-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.532996 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.532946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cnzr\" (UniqueName: \"kubernetes.io/projected/dbaaea17-1a2c-406a-a748-03b901948ead-kube-api-access-4cnzr\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.533613 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.533587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbaaea17-1a2c-406a-a748-03b901948ead-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.535923 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.535875    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbaaea17-1a2c-406a-a748-03b901948ead-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.535923 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.535883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dbaaea17-1a2c-406a-a748-03b901948ead-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.541120 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.541098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cnzr\" (UniqueName: \"kubernetes.io/projected/dbaaea17-1a2c-406a-a748-03b901948ead-kube-api-access-4cnzr\") pod \"prometheus-operator-84d69f794d-qgb82\" (UID: \"dbaaea17-1a2c-406a-a748-03b901948ead\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:51.644329 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:51.644228    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82"
Jul 27 19:28:52.378308 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:52.378284    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qgb82"]
Jul 27 19:28:52.381729 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:52.381703    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbaaea17_1a2c_406a_a748_03b901948ead.slice/crio-f4c19d7413a06ec9ac456280239bfba1704319d4f117943fb02af30c605610a7 WatchSource:0}: Error finding container f4c19d7413a06ec9ac456280239bfba1704319d4f117943fb02af30c605610a7: Status 404 returned error can't find the container with id f4c19d7413a06ec9ac456280239bfba1704319d4f117943fb02af30c605610a7
Jul 27 19:28:52.480544 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:52.480504    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:52.581605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:52.581520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:53.092327 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:53.089319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82" event={"ID":"dbaaea17-1a2c-406a-a748-03b901948ead","Type":"ContainerStarted","Data":"f4c19d7413a06ec9ac456280239bfba1704319d4f117943fb02af30c605610a7"}
Jul 27 19:28:53.093991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:53.093950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5965b89948-nqdwb" event={"ID":"cfba2a95-8524-4c55-bc1f-9c7cc3abc933","Type":"ContainerStarted","Data":"fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9"}
Jul 27 19:28:53.095397 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:53.095371    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568774876c-4f2cf" event={"ID":"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27","Type":"ContainerStarted","Data":"b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1"}
Jul 27 19:28:53.110956 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:53.110907    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5965b89948-nqdwb" podStartSLOduration=2.19394569 podStartE2EDuration="4.110890948s" totalImagesPullingTime="1.916945258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:53.110813142 +0000 UTC m=+185.202164653" watchObservedRunningTime="2026-07-27 19:28:53.110890948 +0000 UTC m=+185.202242446"
Jul 27 19:28:53.127258 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:53.127200    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-568774876c-4f2cf" podStartSLOduration=1.626997014 podStartE2EDuration="5.127188473s" totalImagesPullingTime="3.500191459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:53.126827089 +0000 UTC m=+185.218178587" watchObservedRunningTime="2026-07-27 19:28:53.127188473 +0000 UTC m=+185.218540012"
Jul 27 19:28:54.100457 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:54.100426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82" event={"ID":"dbaaea17-1a2c-406a-a748-03b901948ead","Type":"ContainerStarted","Data":"80f60a6e39b7bb1a118fda68049704a855ffc2cd70f6d232f69e16a782ec5737"}
Jul 27 19:28:54.544683 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:54.544644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:54.645290 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:54.645260    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:54.936873 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:54.936838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:55.042078 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:55.042049    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:55.105313 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:55.105284    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82" event={"ID":"dbaaea17-1a2c-406a-a748-03b901948ead","Type":"ContainerStarted","Data":"d0b94bf5a16d565f4abe561a631784a4db95809e48a03867656f07eb22c2af90"}
Jul 27 19:28:55.172668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:55.172617    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-qgb82" podStartSLOduration=2.527768262 podStartE2EDuration="4.172600783s" totalImagesPullingTime="1.644832521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:55.169341292 +0000 UTC m=+187.260692790" watchObservedRunningTime="2026-07-27 19:28:55.172600783 +0000 UTC m=+187.263952281"
Jul 27 19:28:56.170718 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.170683    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:56.283409 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.283370    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:56.420857 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.420767    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:56.518947 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.518916    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:56.643371 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.643339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:56.666685 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.666653    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"]
Jul 27 19:28:56.669885 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.669867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.672371 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.672313    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 27 19:28:56.672371 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.672355    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xhhr9\""
Jul 27 19:28:56.672513 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.672382    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 27 19:28:56.678704 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.678683    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"]
Jul 27 19:28:56.681667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.681650    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.682489 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.682470    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"]
Jul 27 19:28:56.683981 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.683950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hb8pn\""
Jul 27 19:28:56.683981 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.683977    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 27 19:28:56.684128 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.683988    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 27 19:28:56.684252 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.684223    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 27 19:28:56.701302 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.701274    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"]
Jul 27 19:28:56.702045 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.702023    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rknpk"]
Jul 27 19:28:56.705134 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.705116    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.707621 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.707583    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 19:28:56.707743 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.707585    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 19:28:56.707743 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.707632    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kxzxm\""
Jul 27 19:28:56.707917 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.707904    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 19:28:56.741250 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.741201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:56.779905 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.779868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.780081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.779918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.780081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.779938    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-accelerators-collector-config\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.780081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.779955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-tls\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.780081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-textfile\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.780081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlqqx\" (UniqueName: \"kubernetes.io/projected/fc5733ad-861c-4143-abd6-e88310264296-kube-api-access-tlqqx\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.780081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/95b0add7-044b-44ca-a617-164947d0f3e7-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.780323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95b0add7-044b-44ca-a617-164947d0f3e7-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.780323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-wtmp\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.780323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc5733ad-861c-4143-abd6-e88310264296-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.780323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-sys\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.780323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-root\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.780323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.780323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qjfk\" (UniqueName: \"kubernetes.io/projected/95b0add7-044b-44ca-a617-164947d0f3e7-kube-api-access-6qjfk\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.780323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.780584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.780584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780356    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.780584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e20c8ae-3746-42be-ae69-c9077883b641-metrics-client-ca\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.780584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.780418    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnm5c\" (UniqueName: \"kubernetes.io/projected/1e20c8ae-3746-42be-ae69-c9077883b641-kube-api-access-rnm5c\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.834349 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.834318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:56.881533 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.881497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.881533 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.881534    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.881760 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.881563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.881760 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.881585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e20c8ae-3746-42be-ae69-c9077883b641-metrics-client-ca\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.881760 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:56.881678    2579 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 27 19:28:56.881760 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:56.881741    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-tls podName:95b0add7-044b-44ca-a617-164947d0f3e7 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:57.38172615 +0000 UTC m=+189.473077630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-jrckg" (UID: "95b0add7-044b-44ca-a617-164947d0f3e7") : secret "kube-state-metrics-tls" not found
Jul 27 19:28:56.882015 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.881828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnm5c\" (UniqueName: \"kubernetes.io/projected/1e20c8ae-3746-42be-ae69-c9077883b641-kube-api-access-rnm5c\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882015 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.881901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.882015 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.881955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.882015 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.881987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-accelerators-collector-config\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-tls\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:56.882021    2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-textfile\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:56.882070    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-tls podName:fc5733ad-861c-4143-abd6-e88310264296 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:57.382055935 +0000 UTC m=+189.473407416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-6pnf4" (UID: "fc5733ad-861c-4143-abd6-e88310264296") : secret "openshift-state-metrics-tls" not found
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlqqx\" (UniqueName: \"kubernetes.io/projected/fc5733ad-861c-4143-abd6-e88310264296-kube-api-access-tlqqx\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882139    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e20c8ae-3746-42be-ae69-c9077883b641-metrics-client-ca\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/95b0add7-044b-44ca-a617-164947d0f3e7-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95b0add7-044b-44ca-a617-164947d0f3e7-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.882224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-wtmp\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc5733ad-861c-4143-abd6-e88310264296-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.882668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-sys\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-root\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.882668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-textfile\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qjfk\" (UniqueName: \"kubernetes.io/projected/95b0add7-044b-44ca-a617-164947d0f3e7-kube-api-access-6qjfk\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.882668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882562    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-wtmp\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.882668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-root\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.883076 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.882684    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e20c8ae-3746-42be-ae69-c9077883b641-sys\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.883076 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.883013    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-accelerators-collector-config\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.883268 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.883099    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.883268 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.883116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc5733ad-861c-4143-abd6-e88310264296-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.883268 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.883205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95b0add7-044b-44ca-a617-164947d0f3e7-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.883437 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.883285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/95b0add7-044b-44ca-a617-164947d0f3e7-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.884613 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.884590    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-tls\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.884613 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.884609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.885008 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.884989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e20c8ae-3746-42be-ae69-c9077883b641-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.885044 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.884989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.891609 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.891576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlqqx\" (UniqueName: \"kubernetes.io/projected/fc5733ad-861c-4143-abd6-e88310264296-kube-api-access-tlqqx\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:56.891727 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.891578    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnm5c\" (UniqueName: \"kubernetes.io/projected/1e20c8ae-3746-42be-ae69-c9077883b641-kube-api-access-rnm5c\") pod \"node-exporter-rknpk\" (UID: \"1e20c8ae-3746-42be-ae69-c9077883b641\") " pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:56.891727 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.891682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qjfk\" (UniqueName: \"kubernetes.io/projected/95b0add7-044b-44ca-a617-164947d0f3e7-kube-api-access-6qjfk\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:56.944528 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:56.944491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:57.014586 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.014550    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rknpk"
Jul 27 19:28:57.025782 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:57.025755    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e20c8ae_3746_42be_ae69_c9077883b641.slice/crio-15c2eb5b04f03511e9bcb3180cbbcdf43402192b2d0ce6967cfdb35f92246af0 WatchSource:0}: Error finding container 15c2eb5b04f03511e9bcb3180cbbcdf43402192b2d0ce6967cfdb35f92246af0: Status 404 returned error can't find the container with id 15c2eb5b04f03511e9bcb3180cbbcdf43402192b2d0ce6967cfdb35f92246af0
Jul 27 19:28:57.112698 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.112667    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rknpk" event={"ID":"1e20c8ae-3746-42be-ae69-c9077883b641","Type":"ContainerStarted","Data":"15c2eb5b04f03511e9bcb3180cbbcdf43402192b2d0ce6967cfdb35f92246af0"}
Jul 27 19:28:57.386348 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.386263    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:57.386348 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.386315    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:57.388665 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.388644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b0add7-044b-44ca-a617-164947d0f3e7-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-jrckg\" (UID: \"95b0add7-044b-44ca-a617-164947d0f3e7\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:57.388811 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.388790    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc5733ad-861c-4143-abd6-e88310264296-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6pnf4\" (UID: \"fc5733ad-861c-4143-abd6-e88310264296\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:57.432312 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.432270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:57.531209 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.531169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:57.579183 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.579153    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"
Jul 27 19:28:57.591301 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.591275    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"
Jul 27 19:28:57.666709 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.666388    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:57.736042 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.735965    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4"]
Jul 27 19:28:57.740851 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:57.740429    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5733ad_861c_4143_abd6_e88310264296.slice/crio-9b9b616a1a790c8631ccca4c6fceb737a6c2dfd551f5890031dcec6b8d1ab00f WatchSource:0}: Error finding container 9b9b616a1a790c8631ccca4c6fceb737a6c2dfd551f5890031dcec6b8d1ab00f: Status 404 returned error can't find the container with id 9b9b616a1a790c8631ccca4c6fceb737a6c2dfd551f5890031dcec6b8d1ab00f
Jul 27 19:28:57.758070 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.758042    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg"]
Jul 27 19:28:57.762597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.762576    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:28:57.765359 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.765335    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:57.766619 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.766601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.769237 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.769191    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 27 19:28:57.769237 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.769195    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 27 19:28:57.769237 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.769283    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 27 19:28:57.769237 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.769215    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 27 19:28:57.769605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.769542    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fd9vk\""
Jul 27 19:28:57.769668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.769647    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 27 19:28:57.769772 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.769565    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 27 19:28:57.770259 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.770208    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 27 19:28:57.770445 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.770422    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 27 19:28:57.770966 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.770926    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 27 19:28:57.786870 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.786814    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:28:57.872423 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.872393    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:57.890083 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890003    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnwnd\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-kube-api-access-rnwnd\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890083 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890372 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890372 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890372 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-out\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890372 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890257    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890560 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890560 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890437    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890560 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890526    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890560 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890553    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-web-config\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890731 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890731 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.890731 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.890716    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.982895 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:57.982849    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b0add7_044b_44ca_a617_164947d0f3e7.slice/crio-25a695ca56e1ac96626bf8491bcabc02a6c881158384195207ba518b0d8b18b5 WatchSource:0}: Error finding container 25a695ca56e1ac96626bf8491bcabc02a6c881158384195207ba518b0d8b18b5: Status 404 returned error can't find the container with id 25a695ca56e1ac96626bf8491bcabc02a6c881158384195207ba518b0d8b18b5
Jul 27 19:28:57.992016 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.991956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992016 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.991988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-web-config\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992166 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992166 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992166 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992166 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnwnd\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-kube-api-access-rnwnd\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992509 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992509 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992509 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992509 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-out\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992509 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:28:57.992388    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle podName:d3f68e06-2d39-472c-8c5e-fc15bc9143bb nodeName:}" failed. No retries permitted until 2026-07-27 19:28:58.492364905 +0000 UTC m=+190.583716399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb") : configmap references non-existent config key: ca-bundle.crt
Jul 27 19:28:57.992509 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992509 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992864 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.992864 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.992769    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.994584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.993316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.995626 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.994996    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-out\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.995626 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.995296    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-web-config\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.995626 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.995443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.995821 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.995737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.995976 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.995933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.996138 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.995977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.998044 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.998017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.998131 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.998071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:57.998552 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:57.998528    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:58.000530 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.000504    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnwnd\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-kube-api-access-rnwnd\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:58.117494 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.117461    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg" event={"ID":"95b0add7-044b-44ca-a617-164947d0f3e7","Type":"ContainerStarted","Data":"25a695ca56e1ac96626bf8491bcabc02a6c881158384195207ba518b0d8b18b5"}
Jul 27 19:28:58.119384 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.119359    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4" event={"ID":"fc5733ad-861c-4143-abd6-e88310264296","Type":"ContainerStarted","Data":"8ed8ff57654694128af2a9094ac5f3465927b51a3f560da86d1b9192693d4c99"}
Jul 27 19:28:58.119506 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.119410    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4" event={"ID":"fc5733ad-861c-4143-abd6-e88310264296","Type":"ContainerStarted","Data":"95b4901078197033977aa31fa76711c1a1d6094249994ed8a57c4c7142df9745"}
Jul 27 19:28:58.119506 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.119429    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4" event={"ID":"fc5733ad-861c-4143-abd6-e88310264296","Type":"ContainerStarted","Data":"9b9b616a1a790c8631ccca4c6fceb737a6c2dfd551f5890031dcec6b8d1ab00f"}
Jul 27 19:28:58.120923 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.120899    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rknpk" event={"ID":"1e20c8ae-3746-42be-ae69-c9077883b641","Type":"ContainerStarted","Data":"af9b99572435f23b9d53efb63dfaa3fc8ab7e5402e1fd323bf9ea9802c73f3e6"}
Jul 27 19:28:58.332661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.332623    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:58.499302 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.499265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:58.500200 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.500172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:58.662313 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.662271    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:58.662504 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.662333    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:58.669652 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.669626    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:58.682423 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.682329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:28:58.853171 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.853139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:28:58.928400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:58.928362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:59.009237 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:28:59.009207    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f68e06_2d39_472c_8c5e_fc15bc9143bb.slice/crio-26db440de6bc4e16097a8d46f542dfabe253f25a4c47cdcb0e76a31dfb23f304 WatchSource:0}: Error finding container 26db440de6bc4e16097a8d46f542dfabe253f25a4c47cdcb0e76a31dfb23f304: Status 404 returned error can't find the container with id 26db440de6bc4e16097a8d46f542dfabe253f25a4c47cdcb0e76a31dfb23f304
Jul 27 19:28:59.028738 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.028692    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:59.126900 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.126805    2579 generic.go:356] "Generic (PLEG): container finished" podID="1e20c8ae-3746-42be-ae69-c9077883b641" containerID="af9b99572435f23b9d53efb63dfaa3fc8ab7e5402e1fd323bf9ea9802c73f3e6" exitCode=0
Jul 27 19:28:59.127069 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.126925    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rknpk" event={"ID":"1e20c8ae-3746-42be-ae69-c9077883b641","Type":"ContainerDied","Data":"af9b99572435f23b9d53efb63dfaa3fc8ab7e5402e1fd323bf9ea9802c73f3e6"}
Jul 27 19:28:59.128291 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.128267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerStarted","Data":"26db440de6bc4e16097a8d46f542dfabe253f25a4c47cdcb0e76a31dfb23f304"}
Jul 27 19:28:59.133010 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.132988    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:28:59.648602 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.648497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:59.749876 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.749837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:59.831997 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.831963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:28:59.986959 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:28:59.986926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:00.054783 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.054751    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:29:00.054935 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.054887    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:29:00.059761 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.059741    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:29:00.133702 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.133674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg" event={"ID":"95b0add7-044b-44ca-a617-164947d0f3e7","Type":"ContainerStarted","Data":"a77d2372856a00470c87ca1583c69c0fa051bbc99b70bda24bdb219dfb1e4423"}
Jul 27 19:29:00.133870 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.133735    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg" event={"ID":"95b0add7-044b-44ca-a617-164947d0f3e7","Type":"ContainerStarted","Data":"3922ed43da538676c7ffbb7d4376962eb202214b384cf919d4396a4f5708e198"}
Jul 27 19:29:00.133870 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.133764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg" event={"ID":"95b0add7-044b-44ca-a617-164947d0f3e7","Type":"ContainerStarted","Data":"9aa2b503c5865177b75101095b30a7493a0a689c627afa06be5673dc2306b81d"}
Jul 27 19:29:00.136109 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.136088    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4" event={"ID":"fc5733ad-861c-4143-abd6-e88310264296","Type":"ContainerStarted","Data":"870706c95fd0493ed5c9fc8c09e141a497f5a9d9318c8144f2addee868c286f5"}
Jul 27 19:29:00.138381 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.138354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rknpk" event={"ID":"1e20c8ae-3746-42be-ae69-c9077883b641","Type":"ContainerStarted","Data":"d2e88d64e284d923113bc46376f71623e50da91ed2854b4ece655e387cd8129b"}
Jul 27 19:29:00.138504 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.138404    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rknpk" event={"ID":"1e20c8ae-3746-42be-ae69-c9077883b641","Type":"ContainerStarted","Data":"e9c61f2e8bdbb1efc011233de33ab3dc53021a77426da65e1a34ea3d222bda6b"}
Jul 27 19:29:00.144519 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.144495    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:29:00.155845 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.155785    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jrckg" podStartSLOduration=2.765702974 podStartE2EDuration="4.155765691s" totalImagesPullingTime="1.390062717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:00.155613917 +0000 UTC m=+192.246965402" watchObservedRunningTime="2026-07-27 19:29:00.155765691 +0000 UTC m=+192.247117183"
Jul 27 19:29:00.214934 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.214882    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rknpk" podStartSLOduration=3.141772094 podStartE2EDuration="4.214865867s" totalImagesPullingTime="1.006339145s" totalInitContainerRuntime="66.754628ms" isStatefulPod=true podCreationTimestamp="2026-07-27 19:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:00.213574547 +0000 UTC m=+192.304926045" watchObservedRunningTime="2026-07-27 19:29:00.214865867 +0000 UTC m=+192.306217364"
Jul 27 19:29:00.232931 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.232899    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568774876c-4f2cf"]
Jul 27 19:29:00.239846 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.239762    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pnf4" podStartSLOduration=2.921073792 podStartE2EDuration="4.239747473s" totalImagesPullingTime="1.318673681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:00.239136406 +0000 UTC m=+192.330487905" watchObservedRunningTime="2026-07-27 19:29:00.239747473 +0000 UTC m=+192.331098949"
Jul 27 19:29:00.370118 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.370071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:00.423036 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.422998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:00.735141 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.735112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:00.819694 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.819662    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"]
Jul 27 19:29:00.823158 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.823137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:00.825868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.825817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 27 19:29:00.825868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.825833    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 27 19:29:00.825868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.825850    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 27 19:29:00.826134 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.825893    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qs565\""
Jul 27 19:29:00.826134 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.825929    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 27 19:29:00.826834 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.826814    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-aidjngvlab44i\""
Jul 27 19:29:00.826967 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.826860    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 27 19:29:00.837899 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.837873    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"]
Jul 27 19:29:00.927819 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.927786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-grpc-tls\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:00.928006 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.927828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-tls\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:00.928006 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.927858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:00.928006 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.927885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:00.928006 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.927916    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa44b356-c08d-49ab-997c-f61420b24564-metrics-client-ca\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:00.928006 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.927934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:00.928006 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.927974    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2pjp\" (UniqueName: \"kubernetes.io/projected/fa44b356-c08d-49ab-997c-f61420b24564-kube-api-access-g2pjp\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:00.928231 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:00.928023    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.012610 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.012530    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-854b86fcd8-mxfbr"]
Jul 27 19:29:01.015810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.015787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.018235 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.018211    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 27 19:29:01.018385 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.018361    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 27 19:29:01.018385 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.018369    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 27 19:29:01.018527 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.018384    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 27 19:29:01.018527 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.018448    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7fdk589fi4r7b\""
Jul 27 19:29:01.018527 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.018497    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q6cvq\""
Jul 27 19:29:01.026097 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.026074    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-854b86fcd8-mxfbr"]
Jul 27 19:29:01.029067 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.029040    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-grpc-tls\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.029184 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.029090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-tls\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.029184 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.029132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.029184 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.029161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.029386 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.029215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa44b356-c08d-49ab-997c-f61420b24564-metrics-client-ca\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.029386 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.029273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.029386 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.029301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2pjp\" (UniqueName: \"kubernetes.io/projected/fa44b356-c08d-49ab-997c-f61420b24564-kube-api-access-g2pjp\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.029386 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.029378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.030154 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.030005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa44b356-c08d-49ab-997c-f61420b24564-metrics-client-ca\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.032015 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.031966    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.032102 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.032043    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-grpc-tls\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.032262 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.032218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.032442 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.032409    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.032680 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.032661    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-tls\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.032741 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.032674    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa44b356-c08d-49ab-997c-f61420b24564-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.038364 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.038341    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2pjp\" (UniqueName: \"kubernetes.io/projected/fa44b356-c08d-49ab-997c-f61420b24564-kube-api-access-g2pjp\") pod \"thanos-querier-66bc687cfd-5xxbj\" (UID: \"fa44b356-c08d-49ab-997c-f61420b24564\") " pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.130718 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.130676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d5780864-eb8b-462d-9bcc-0b53d408bd52-audit-log\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.130888 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.130723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d5780864-eb8b-462d-9bcc-0b53d408bd52-metrics-server-audit-profiles\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.130888 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.130749    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkjvw\" (UniqueName: \"kubernetes.io/projected/d5780864-eb8b-462d-9bcc-0b53d408bd52-kube-api-access-mkjvw\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.130888 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.130818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-secret-metrics-server-tls\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.130991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.130887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-secret-metrics-server-client-certs\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.130991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.130913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5780864-eb8b-462d-9bcc-0b53d408bd52-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.131057 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.130988    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-client-ca-bundle\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.132622 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.132599    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:01.143329 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.143301    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerID="285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754" exitCode=0
Jul 27 19:29:01.143465 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.143400    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerDied","Data":"285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754"}
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.238663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d5780864-eb8b-462d-9bcc-0b53d408bd52-metrics-server-audit-profiles\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.239377    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkjvw\" (UniqueName: \"kubernetes.io/projected/d5780864-eb8b-462d-9bcc-0b53d408bd52-kube-api-access-mkjvw\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.239457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-secret-metrics-server-tls\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.239715    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-secret-metrics-server-client-certs\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.239730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d5780864-eb8b-462d-9bcc-0b53d408bd52-metrics-server-audit-profiles\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.239764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5780864-eb8b-462d-9bcc-0b53d408bd52-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.240058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-client-ca-bundle\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.240109    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d5780864-eb8b-462d-9bcc-0b53d408bd52-audit-log\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.244941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5780864-eb8b-462d-9bcc-0b53d408bd52-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.245612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.245405    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d5780864-eb8b-462d-9bcc-0b53d408bd52-audit-log\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.248388 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.247827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-client-ca-bundle\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.254972 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.254915    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-secret-metrics-server-client-certs\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.255109 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.255070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d5780864-eb8b-462d-9bcc-0b53d408bd52-secret-metrics-server-tls\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.255723 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.255703    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkjvw\" (UniqueName: \"kubernetes.io/projected/d5780864-eb8b-462d-9bcc-0b53d408bd52-kube-api-access-mkjvw\") pod \"metrics-server-854b86fcd8-mxfbr\" (UID: \"d5780864-eb8b-462d-9bcc-0b53d408bd52\") " pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.295515 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.295390    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"]
Jul 27 19:29:01.297194 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:29:01.297153    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa44b356_c08d_49ab_997c_f61420b24564.slice/crio-97dac1b3bc02436595a61a68e8257f4c208ac5ea2637eed85fbe9ff3b73f48b8 WatchSource:0}: Error finding container 97dac1b3bc02436595a61a68e8257f4c208ac5ea2637eed85fbe9ff3b73f48b8: Status 404 returned error can't find the container with id 97dac1b3bc02436595a61a68e8257f4c208ac5ea2637eed85fbe9ff3b73f48b8
Jul 27 19:29:01.325129 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.325102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:01.458517 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.458495    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-854b86fcd8-mxfbr"]
Jul 27 19:29:01.460768 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:29:01.460741    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5780864_eb8b_462d_9bcc_0b53d408bd52.slice/crio-79c9f6332671d1cfdc6a0be5e4663c6591ab28504432f1e90a4e6b298b5c6a50 WatchSource:0}: Error finding container 79c9f6332671d1cfdc6a0be5e4663c6591ab28504432f1e90a4e6b298b5c6a50: Status 404 returned error can't find the container with id 79c9f6332671d1cfdc6a0be5e4663c6591ab28504432f1e90a4e6b298b5c6a50
Jul 27 19:29:01.465234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.465203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:01.567908 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.567811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:01.666581 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.666540    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:01.767297 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.767221    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:01.961939 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:01.961895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:02.061439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.061395    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:02.153085 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.153045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr" event={"ID":"d5780864-eb8b-462d-9bcc-0b53d408bd52","Type":"ContainerStarted","Data":"79c9f6332671d1cfdc6a0be5e4663c6591ab28504432f1e90a4e6b298b5c6a50"}
Jul 27 19:29:02.155578 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.155543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj" event={"ID":"fa44b356-c08d-49ab-997c-f61420b24564","Type":"ContainerStarted","Data":"97dac1b3bc02436595a61a68e8257f4c208ac5ea2637eed85fbe9ff3b73f48b8"}
Jul 27 19:29:02.920110 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.920054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:02.928806 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.926508    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:29:02.933174 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.933147    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:02.936041 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.936012    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 27 19:29:02.936169 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.936131    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fkckc9e9fot2m\""
Jul 27 19:29:02.936256 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.936173    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 27 19:29:02.936323 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.936261    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 27 19:29:02.936934 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.936557    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 27 19:29:02.936934 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.936610    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nh2bt\""
Jul 27 19:29:02.936934 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.936561    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 27 19:29:02.937137 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.937007    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 27 19:29:02.937137 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.937106    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 27 19:29:02.937263 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.937216    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 27 19:29:02.937669 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.937438    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 27 19:29:02.937669 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.937465    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 27 19:29:02.938124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.937902    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 27 19:29:02.938660 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.938378    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 27 19:29:02.939350 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.939320    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 27 19:29:02.947930 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.947908    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 27 19:29:02.950041 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:02.950020    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:29:03.058980 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.058940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059176 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.058991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brgrv\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-kube-api-access-brgrv\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059176 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059176 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059176 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059444 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059444 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059314    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059444 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059372    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059444 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059634 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059793 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.059968 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.060052 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.059998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.060113 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.060061    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.060165 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.060117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.060225 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.060176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.060225 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.060207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.060359 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.060227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.060359 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.060314    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.060359 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.060331    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161334 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161298    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brgrv\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-kube-api-access-brgrv\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161586    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161714    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.161775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161784    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.161980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.162008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162901 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.162625    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.162901 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.162691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.164821 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.164794    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.169234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.165388    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.169234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.165907    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.169234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.166522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.169234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.168034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.169234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.168373    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.169234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.168839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.169637 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.169300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.170610 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.170525    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.170712 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.170616    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.171576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.171126    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.171576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.171306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.171576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.171521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.171937 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.171859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brgrv\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-kube-api-access-brgrv\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.171937 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.171888    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.172209 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.172183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.172931 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.172906    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.255612 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.255556    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:03.882874 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:03.882837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:04.635431 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:04.635326    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:29:04.639435 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:29:04.639357    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f52bcd5_25d1_450d_ac64_68cdc5c46262.slice/crio-2ebc1b4d28219c3f559929f7c82fd4be5557509c3c2dad603c09a816dbf5144a WatchSource:0}: Error finding container 2ebc1b4d28219c3f559929f7c82fd4be5557509c3c2dad603c09a816dbf5144a: Status 404 returned error can't find the container with id 2ebc1b4d28219c3f559929f7c82fd4be5557509c3c2dad603c09a816dbf5144a
Jul 27 19:29:05.171807 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.171761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerStarted","Data":"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e"}
Jul 27 19:29:05.172024 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.171853    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerStarted","Data":"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b"}
Jul 27 19:29:05.172024 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.171886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerStarted","Data":"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8"}
Jul 27 19:29:05.172024 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.171912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerStarted","Data":"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743"}
Jul 27 19:29:05.172024 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.171941    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerStarted","Data":"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903"}
Jul 27 19:29:05.173300 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.173278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr" event={"ID":"d5780864-eb8b-462d-9bcc-0b53d408bd52","Type":"ContainerStarted","Data":"373e32fa72847104022fc30dda9ebc1dbe2a8314f32f22b68c68829eeb73b50d"}
Jul 27 19:29:05.175214 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.175192    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj" event={"ID":"fa44b356-c08d-49ab-997c-f61420b24564","Type":"ContainerStarted","Data":"d8f34eb914ae8c5c2b5db0fff58ab3e48c761fd9bbf6990786cbb2b4dc25a84a"}
Jul 27 19:29:05.175341 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.175274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj" event={"ID":"fa44b356-c08d-49ab-997c-f61420b24564","Type":"ContainerStarted","Data":"a80e55f2d1b54f730613fac208ec16ef9baf5658d8a88e6618f2b9f6e8d7cb43"}
Jul 27 19:29:05.175341 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.175316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj" event={"ID":"fa44b356-c08d-49ab-997c-f61420b24564","Type":"ContainerStarted","Data":"2b9b0de3d6f85adb75cfd2d8985a3f6cc4ef2ba3118b8620cc55948f61ad413b"}
Jul 27 19:29:05.176883 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.176853    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerID="5fb03e878bd3ca609ccf499894dab28a5a53adb413238bafc9415a5e8b6f044b" exitCode=0
Jul 27 19:29:05.176986 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.176889    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerDied","Data":"5fb03e878bd3ca609ccf499894dab28a5a53adb413238bafc9415a5e8b6f044b"}
Jul 27 19:29:05.176986 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.176946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerStarted","Data":"2ebc1b4d28219c3f559929f7c82fd4be5557509c3c2dad603c09a816dbf5144a"}
Jul 27 19:29:05.192115 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:05.192073    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr" podStartSLOduration=2.169295005 podStartE2EDuration="5.192058238s" totalImagesPullingTime="3.022763233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:05.19042192 +0000 UTC m=+197.281773419" watchObservedRunningTime="2026-07-27 19:29:05.192058238 +0000 UTC m=+197.283409715"
Jul 27 19:29:07.191145 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:07.191108    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj" event={"ID":"fa44b356-c08d-49ab-997c-f61420b24564","Type":"ContainerStarted","Data":"32f0d03e38270d94f2d7b9032741214435e511cc93669ba1988ebaaeb79b7ecd"}
Jul 27 19:29:07.191690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:07.191192    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj" event={"ID":"fa44b356-c08d-49ab-997c-f61420b24564","Type":"ContainerStarted","Data":"ad79f07bd2df5ba0591596511796807e4b8af9e9bcfebc0de50fd9963fd9575e"}
Jul 27 19:29:07.191690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:07.191257    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:07.191690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:07.191300    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj" event={"ID":"fa44b356-c08d-49ab-997c-f61420b24564","Type":"ContainerStarted","Data":"27dc1d8a3557e25145131497627dc3beb246e342b079a2bfd0726c8b05b0f54b"}
Jul 27 19:29:07.194730 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:07.194688    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerStarted","Data":"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb"}
Jul 27 19:29:07.214047 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:07.213982    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj" podStartSLOduration=2.419327263 podStartE2EDuration="7.213963456s" totalImagesPullingTime="4.794636193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:07.212948652 +0000 UTC m=+199.304300188" watchObservedRunningTime="2026-07-27 19:29:07.213963456 +0000 UTC m=+199.305314954"
Jul 27 19:29:07.240514 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:07.240461    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.565574368 podStartE2EDuration="10.240443567s" totalImagesPullingTime="5.659251159s" totalInitContainerRuntime="15.61804ms" isStatefulPod=false podCreationTimestamp="2026-07-27 19:28:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:07.238272366 +0000 UTC m=+199.329623866" watchObservedRunningTime="2026-07-27 19:29:07.240443567 +0000 UTC m=+199.331795064"
Jul 27 19:29:09.208327 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:09.208286    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerStarted","Data":"107c6beec2a1e1a8de6542fee491ec91638679457bf5f25c20cd779ab3ad8b03"}
Jul 27 19:29:09.208701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:09.208358    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerStarted","Data":"b97af28f7fdae0c1dc7d577496fd77b991a1757a7f6e8f4512c79bf875db31e2"}
Jul 27 19:29:09.208701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:09.208391    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerStarted","Data":"e866943e85aa7598d9f857d3534a7714b79fc30907a1c79fd82204c7ac329543"}
Jul 27 19:29:09.208701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:09.208429    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerStarted","Data":"56781ce410983caf39c23adcc9793a551616c88bbee646905305b80a793aaf3b"}
Jul 27 19:29:09.208701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:09.208458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerStarted","Data":"3103a2568515159f5e8ea36624fbb84f19689f0842679ccf0a08b336b7c2ee2d"}
Jul 27 19:29:09.208701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:09.208481    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerStarted","Data":"624f6db288f3d6988364e20c90f9450ce132b10844212cc197f71d3c07f29a76"}
Jul 27 19:29:09.240327 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:09.240276    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.970780553 podStartE2EDuration="7.240259003s" totalImagesPullingTime="3.238153282s" totalInitContainerRuntime="31.325168ms" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:09.238030871 +0000 UTC m=+201.329382370" watchObservedRunningTime="2026-07-27 19:29:09.240259003 +0000 UTC m=+201.331610498"
Jul 27 19:29:10.313400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:10.313360    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:10.384237 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:10.384194    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:13.208451 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:13.208422    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66bc687cfd-5xxbj"
Jul 27 19:29:13.256587 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:13.256552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:29:14.356304 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:14.356259    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5965b89948-nqdwb"]
Jul 27 19:29:14.459714 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:14.459681    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:14.562159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:14.562123    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:14.635217 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:14.635116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:16.118137 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:16.118102    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:16.217786 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:16.217741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:16.320685 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:16.320644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:16.382307 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:16.382195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:21.326012 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:21.325972    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:21.326416 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:21.326043    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:25.648590 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:25.648545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:25.717829 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:25.717793    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:26.163508 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.163465    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-568774876c-4f2cf" podUID="f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" containerName="console" containerID="cri-o://b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1" gracePeriod=15
Jul 27 19:29:26.409414 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.409389    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:29:26.479081 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479043    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-oauth-serving-cert\") pod \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") "
Jul 27 19:29:26.479316 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479114    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66gtl\" (UniqueName: \"kubernetes.io/projected/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-kube-api-access-66gtl\") pod \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") "
Jul 27 19:29:26.479316 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479144    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-serving-cert\") pod \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") "
Jul 27 19:29:26.479316 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479268    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-service-ca\") pod \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") "
Jul 27 19:29:26.479503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479341    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-config\") pod \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") "
Jul 27 19:29:26.479503 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479373    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-oauth-config\") pod \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\" (UID: \"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27\") "
Jul 27 19:29:26.479628 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479572    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" (UID: "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:26.479695 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479626    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-service-ca" (OuterVolumeSpecName: "service-ca") pod "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" (UID: "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:26.479748 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479702    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-config" (OuterVolumeSpecName: "console-config") pod "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" (UID: "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:26.479801 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479769    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-oauth-serving-cert\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:26.479801 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.479790    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:26.481515 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.481485    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-kube-api-access-66gtl" (OuterVolumeSpecName: "kube-api-access-66gtl") pod "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" (UID: "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27"). InnerVolumeSpecName "kube-api-access-66gtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:29:26.481647 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.481578    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" (UID: "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:29:26.481647 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.481604    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" (UID: "f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:29:26.581233 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.581197    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-config\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:26.581233 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.581225    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-oauth-config\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:26.581233 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.581237    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66gtl\" (UniqueName: \"kubernetes.io/projected/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-kube-api-access-66gtl\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:26.581233 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:26.581267    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27-console-serving-cert\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:27.281194 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.281157    2579 generic.go:356] "Generic (PLEG): container finished" podID="f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" containerID="b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1" exitCode=0
Jul 27 19:29:27.281741 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.281206    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568774876c-4f2cf"
Jul 27 19:29:27.281741 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.281269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568774876c-4f2cf" event={"ID":"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27","Type":"ContainerDied","Data":"b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1"}
Jul 27 19:29:27.281741 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.281321    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568774876c-4f2cf" event={"ID":"f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27","Type":"ContainerDied","Data":"ddcbbc8b7d57a1a67f1b0a02aabc2196bfb32e31400002aa41050168d643a16d"}
Jul 27 19:29:27.281741 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.281343    2579 scope.go:117] "RemoveContainer" containerID="b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1"
Jul 27 19:29:27.289671 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.289655    2579 scope.go:117] "RemoveContainer" containerID="b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1"
Jul 27 19:29:27.289965 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:29:27.289942    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1\": container with ID starting with b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1 not found: ID does not exist" containerID="b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1"
Jul 27 19:29:27.290037 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.289973    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1"} err="failed to get container status \"b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1\": rpc error: code = NotFound desc = could not find container \"b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1\": container with ID starting with b6a9cc56bd542cd92b2fa6c00f862c92ee4315ec7814b943b33c1d0ed61268a1 not found: ID does not exist"
Jul 27 19:29:27.296918 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.296891    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568774876c-4f2cf"]
Jul 27 19:29:27.301510 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:27.301487    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-568774876c-4f2cf"]
Jul 27 19:29:28.487590 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:28.487557    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" path="/var/lib/kubelet/pods/f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27/volumes"
Jul 27 19:29:32.653740 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:32.653703    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:39.376803 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.376736    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5965b89948-nqdwb" podUID="cfba2a95-8524-4c55-bc1f-9c7cc3abc933" containerName="console" containerID="cri-o://fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9" gracePeriod=15
Jul 27 19:29:39.622282 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.622236    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:29:39.703596 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.703563    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-trusted-ca-bundle\") pod \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") "
Jul 27 19:29:39.703785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.703607    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-config\") pod \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") "
Jul 27 19:29:39.703785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.703640    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-service-ca\") pod \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") "
Jul 27 19:29:39.703785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.703680    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-serving-cert\") pod \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") "
Jul 27 19:29:39.703785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.703720    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-oauth-config\") pod \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") "
Jul 27 19:29:39.703785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.703764    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfqkh\" (UniqueName: \"kubernetes.io/projected/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-kube-api-access-pfqkh\") pod \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") "
Jul 27 19:29:39.704063 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.703791    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-oauth-serving-cert\") pod \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\" (UID: \"cfba2a95-8524-4c55-bc1f-9c7cc3abc933\") "
Jul 27 19:29:39.704063 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.704043    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cfba2a95-8524-4c55-bc1f-9c7cc3abc933" (UID: "cfba2a95-8524-4c55-bc1f-9c7cc3abc933"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:39.704167 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.704057    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-config" (OuterVolumeSpecName: "console-config") pod "cfba2a95-8524-4c55-bc1f-9c7cc3abc933" (UID: "cfba2a95-8524-4c55-bc1f-9c7cc3abc933"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:39.704259 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.704210    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-service-ca" (OuterVolumeSpecName: "service-ca") pod "cfba2a95-8524-4c55-bc1f-9c7cc3abc933" (UID: "cfba2a95-8524-4c55-bc1f-9c7cc3abc933"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:39.704318 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.704300    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-trusted-ca-bundle\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:39.704456 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.704432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cfba2a95-8524-4c55-bc1f-9c7cc3abc933" (UID: "cfba2a95-8524-4c55-bc1f-9c7cc3abc933"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:39.705975 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.705938    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cfba2a95-8524-4c55-bc1f-9c7cc3abc933" (UID: "cfba2a95-8524-4c55-bc1f-9c7cc3abc933"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:29:39.705975 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.705965    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-kube-api-access-pfqkh" (OuterVolumeSpecName: "kube-api-access-pfqkh") pod "cfba2a95-8524-4c55-bc1f-9c7cc3abc933" (UID: "cfba2a95-8524-4c55-bc1f-9c7cc3abc933"). InnerVolumeSpecName "kube-api-access-pfqkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:29:39.706112 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.705985    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cfba2a95-8524-4c55-bc1f-9c7cc3abc933" (UID: "cfba2a95-8524-4c55-bc1f-9c7cc3abc933"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:29:39.805414 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.805365    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-oauth-config\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:39.805414 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.805407    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfqkh\" (UniqueName: \"kubernetes.io/projected/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-kube-api-access-pfqkh\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:39.805414 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.805417    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-oauth-serving-cert\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:39.805414 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.805426    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-config\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:39.805414 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.805434    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:39.805727 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:39.805442    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfba2a95-8524-4c55-bc1f-9c7cc3abc933-console-serving-cert\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.334587 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.334553    2579 generic.go:356] "Generic (PLEG): container finished" podID="cfba2a95-8524-4c55-bc1f-9c7cc3abc933" containerID="fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9" exitCode=0
Jul 27 19:29:40.334764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.334598    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5965b89948-nqdwb"
Jul 27 19:29:40.334764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.334640    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5965b89948-nqdwb" event={"ID":"cfba2a95-8524-4c55-bc1f-9c7cc3abc933","Type":"ContainerDied","Data":"fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9"}
Jul 27 19:29:40.334764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.334689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5965b89948-nqdwb" event={"ID":"cfba2a95-8524-4c55-bc1f-9c7cc3abc933","Type":"ContainerDied","Data":"be5d0a7ddeafb066576c96942539bb44724ebe323d314096652f3cf8f2095cce"}
Jul 27 19:29:40.334764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.334713    2579 scope.go:117] "RemoveContainer" containerID="fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9"
Jul 27 19:29:40.343150 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.343129    2579 scope.go:117] "RemoveContainer" containerID="fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9"
Jul 27 19:29:40.343432 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:29:40.343410    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9\": container with ID starting with fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9 not found: ID does not exist" containerID="fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9"
Jul 27 19:29:40.343487 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.343440    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9"} err="failed to get container status \"fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9\": rpc error: code = NotFound desc = could not find container \"fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9\": container with ID starting with fa673e302b8cb0cc8645e72e8e3460b4be3ad9057167b4b6c8ffe2661fb611f9 not found: ID does not exist"
Jul 27 19:29:40.354572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.354549    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5965b89948-nqdwb"]
Jul 27 19:29:40.358966 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.358945    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5965b89948-nqdwb"]
Jul 27 19:29:40.487824 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.487792    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfba2a95-8524-4c55-bc1f-9c7cc3abc933" path="/var/lib/kubelet/pods/cfba2a95-8524-4c55-bc1f-9c7cc3abc933/volumes"
Jul 27 19:29:40.631849 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.631760    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:40.690033 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:40.689992    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:41.330417 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:41.330386    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:41.334238 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:41.334217    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-854b86fcd8-mxfbr"
Jul 27 19:29:55.145904 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:55.145867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:55.217169 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:55.217128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:57.580421 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:57.580374    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:29:57.647844 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:29:57.647814    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:00.299599 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.299544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:30:00.299599 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.299605    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:30:00.302598 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.302575    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 19:30:00.302729 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.302601    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 19:30:00.312204 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.312176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1681bd39-34f5-4469-8ed5-fc75330bad04-metrics-certs\") pod \"network-metrics-daemon-qc4zf\" (UID: \"1681bd39-34f5-4469-8ed5-fc75330bad04\") " pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:30:00.312312 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.312261    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/185be14c-e7d0-494b-b626-94f381a9ab93-original-pull-secret\") pod \"global-pull-secret-syncer-wtrmw\" (UID: \"185be14c-e7d0-494b-b626-94f381a9ab93\") " pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:30:00.400562 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.400509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:30:00.403463 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.403445    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 19:30:00.414033 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.414008    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 19:30:00.424601 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.424576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzf8n\" (UniqueName: \"kubernetes.io/projected/9f4eccf9-761d-4834-8f82-d244d41d38c7-kube-api-access-zzf8n\") pod \"network-check-target-9km82\" (UID: \"9f4eccf9-761d-4834-8f82-d244d41d38c7\") " pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:30:00.495602 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.495569    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xt9mb\""
Jul 27 19:30:00.498782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.498766    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wtrmw"
Jul 27 19:30:00.503196 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.503180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:30:00.505536 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.505521    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4682l\""
Jul 27 19:30:00.513944 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.513709    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qc4zf"
Jul 27 19:30:00.673879 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.673855    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wtrmw"]
Jul 27 19:30:00.676547 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:30:00.676521    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185be14c_e7d0_494b_b626_94f381a9ab93.slice/crio-b162ae3c1a66015ab16f00c4c7e627f00c06c5b23284ff6edf0b66bc2faead76 WatchSource:0}: Error finding container b162ae3c1a66015ab16f00c4c7e627f00c06c5b23284ff6edf0b66bc2faead76: Status 404 returned error can't find the container with id b162ae3c1a66015ab16f00c4c7e627f00c06c5b23284ff6edf0b66bc2faead76
Jul 27 19:30:00.680812 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.680789    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9km82"]
Jul 27 19:30:00.682631 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:30:00.682609    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4eccf9_761d_4834_8f82_d244d41d38c7.slice/crio-c333a7460e65af3efafa371101eff69fbde364dd27f4898251b01bbff2ec5ff2 WatchSource:0}: Error finding container c333a7460e65af3efafa371101eff69fbde364dd27f4898251b01bbff2ec5ff2: Status 404 returned error can't find the container with id c333a7460e65af3efafa371101eff69fbde364dd27f4898251b01bbff2ec5ff2
Jul 27 19:30:00.687996 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:00.687973    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qc4zf"]
Jul 27 19:30:00.690451 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:30:00.690429    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1681bd39_34f5_4469_8ed5_fc75330bad04.slice/crio-7e9f31c95c9340684879b717779b7f5b86a0454c5108eaaad3b098354d5e39ca WatchSource:0}: Error finding container 7e9f31c95c9340684879b717779b7f5b86a0454c5108eaaad3b098354d5e39ca: Status 404 returned error can't find the container with id 7e9f31c95c9340684879b717779b7f5b86a0454c5108eaaad3b098354d5e39ca
Jul 27 19:30:01.423037 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:01.422983    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qc4zf" event={"ID":"1681bd39-34f5-4469-8ed5-fc75330bad04","Type":"ContainerStarted","Data":"7e9f31c95c9340684879b717779b7f5b86a0454c5108eaaad3b098354d5e39ca"}
Jul 27 19:30:01.424526 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:01.424489    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wtrmw" event={"ID":"185be14c-e7d0-494b-b626-94f381a9ab93","Type":"ContainerStarted","Data":"b162ae3c1a66015ab16f00c4c7e627f00c06c5b23284ff6edf0b66bc2faead76"}
Jul 27 19:30:01.425843 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:01.425818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9km82" event={"ID":"9f4eccf9-761d-4834-8f82-d244d41d38c7","Type":"ContainerStarted","Data":"c333a7460e65af3efafa371101eff69fbde364dd27f4898251b01bbff2ec5ff2"}
Jul 27 19:30:02.432474 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:02.432435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qc4zf" event={"ID":"1681bd39-34f5-4469-8ed5-fc75330bad04","Type":"ContainerStarted","Data":"c0dd8e69a2c7bcfd16ee63cfcd618cdb246f0ac8709f3599a4f4b63428933d59"}
Jul 27 19:30:02.432958 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:02.432544    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qc4zf" event={"ID":"1681bd39-34f5-4469-8ed5-fc75330bad04","Type":"ContainerStarted","Data":"5190926840af8837c69c69e889c95fc43c26956cf9ebe4e38347bce0dbf62cac"}
Jul 27 19:30:02.450257 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:02.450185    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qc4zf" podStartSLOduration=252.375999607 podStartE2EDuration="4m13.45016257s" totalImagesPullingTime="1.074162963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:02.448373993 +0000 UTC m=+254.539725493" watchObservedRunningTime="2026-07-27 19:30:02.45016257 +0000 UTC m=+254.541514071"
Jul 27 19:30:02.548068 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:02.548031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:02.601814 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:02.601771    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:03.256737 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:03.256689    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:03.280534 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:03.280505    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:03.454099 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:03.454066    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:04.442656 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:04.442623    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9km82" event={"ID":"9f4eccf9-761d-4834-8f82-d244d41d38c7","Type":"ContainerStarted","Data":"d145f9bcf09188eadacb6e15e73662483b7690c84ef72d73a58f23eb7f958820"}
Jul 27 19:30:04.443068 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:04.443053    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:30:04.459001 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:04.458954    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9km82" podStartSLOduration=252.54245877 podStartE2EDuration="4m15.458938949s" totalImagesPullingTime="2.916480179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:04.457688516 +0000 UTC m=+256.549040016" watchObservedRunningTime="2026-07-27 19:30:04.458938949 +0000 UTC m=+256.550290447"
Jul 27 19:30:09.467845 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:09.467803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wtrmw" event={"ID":"185be14c-e7d0-494b-b626-94f381a9ab93","Type":"ContainerStarted","Data":"e0b761b63ce25253fef9152b19c9b0159da012f2a914f4b353a16100bb228206"}
Jul 27 19:30:09.483023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:09.482968    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wtrmw" podStartSLOduration=252.03821045 podStartE2EDuration="4m20.482952828s" totalImagesPullingTime="8.444742378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:09.48267752 +0000 UTC m=+261.574029018" watchObservedRunningTime="2026-07-27 19:30:09.482952828 +0000 UTC m=+261.574304350"
Jul 27 19:30:15.937574 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:15.937541    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:16.037399 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:16.037366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:16.986274 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:16.986223    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:16.986710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:16.986524    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="alertmanager" containerID="cri-o://9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903" gracePeriod=120
Jul 27 19:30:16.986710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:16.986600    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy-metric" containerID="cri-o://2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e" gracePeriod=120
Jul 27 19:30:16.986710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:16.986655    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy-web" containerID="cri-o://10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8" gracePeriod=120
Jul 27 19:30:16.986710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:16.986601    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy" containerID="cri-o://6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b" gracePeriod=120
Jul 27 19:30:16.986710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:16.986659    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="config-reloader" containerID="cri-o://882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743" gracePeriod=120
Jul 27 19:30:16.986951 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:16.986665    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="prom-label-proxy" containerID="cri-o://333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb" gracePeriod=120
Jul 27 19:30:17.501854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:17.501816    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerID="333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb" exitCode=0
Jul 27 19:30:17.501854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:17.501841    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerID="6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b" exitCode=0
Jul 27 19:30:17.501854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:17.501847    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerID="882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743" exitCode=0
Jul 27 19:30:17.501854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:17.501853    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerID="9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903" exitCode=0
Jul 27 19:30:17.502121 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:17.501888    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerDied","Data":"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb"}
Jul 27 19:30:17.502121 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:17.501949    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerDied","Data":"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b"}
Jul 27 19:30:17.502121 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:17.501976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerDied","Data":"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743"}
Jul 27 19:30:17.502121 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:17.502000    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerDied","Data":"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903"}
Jul 27 19:30:18.334382 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.334317    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.462160 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462124    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462160 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462164    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-cluster-tls-config\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462196    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462296    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-main-tls\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462346    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-out\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462366    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-volume\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462392    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462420    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-main-db\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462767 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462732    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:18.462828 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462786    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-web-config\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462879 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462867    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-metrics-client-ca\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462934 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462902    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462988 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-tls-assets\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.462988 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.462978    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnwnd\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-kube-api-access-rnwnd\") pod \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\" (UID: \"d3f68e06-2d39-472c-8c5e-fc15bc9143bb\") "
Jul 27 19:30:18.463578 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.463332    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-main-db\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.463578 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.463340    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:18.463578 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.463408    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:18.465603 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.465564    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:18.465905 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.465870    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-out" (OuterVolumeSpecName: "config-out") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:18.466017 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.465931    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:18.466197 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.466169    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:18.466358 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.466332    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-kube-api-access-rnwnd" (OuterVolumeSpecName: "kube-api-access-rnwnd") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "kube-api-access-rnwnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:18.466463 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.466351    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-volume" (OuterVolumeSpecName: "config-volume") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:18.466546 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.466530    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:18.467108 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.467086    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:18.469660 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.469634    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:18.476335 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.476304    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-web-config" (OuterVolumeSpecName: "web-config") pod "d3f68e06-2d39-472c-8c5e-fc15bc9143bb" (UID: "d3f68e06-2d39-472c-8c5e-fc15bc9143bb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:18.508492 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.508462    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerID="2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e" exitCode=0
Jul 27 19:30:18.508492 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.508487    2579 generic.go:356] "Generic (PLEG): container finished" podID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerID="10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8" exitCode=0
Jul 27 19:30:18.508660 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.508532    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.508660 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.508537    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerDied","Data":"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e"}
Jul 27 19:30:18.508660 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.508606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerDied","Data":"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8"}
Jul 27 19:30:18.508660 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.508635    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3f68e06-2d39-472c-8c5e-fc15bc9143bb","Type":"ContainerDied","Data":"26db440de6bc4e16097a8d46f542dfabe253f25a4c47cdcb0e76a31dfb23f304"}
Jul 27 19:30:18.508810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.508673    2579 scope.go:117] "RemoveContainer" containerID="333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb"
Jul 27 19:30:18.516832 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.516812    2579 scope.go:117] "RemoveContainer" containerID="2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e"
Jul 27 19:30:18.524148 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.524132    2579 scope.go:117] "RemoveContainer" containerID="6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b"
Jul 27 19:30:18.531102 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.531078    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:18.532542 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.532449    2579 scope.go:117] "RemoveContainer" containerID="10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8"
Jul 27 19:30:18.534262 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.534223    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:18.540316 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.540295    2579 scope.go:117] "RemoveContainer" containerID="882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743"
Jul 27 19:30:18.547868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.547846    2579 scope.go:117] "RemoveContainer" containerID="9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903"
Jul 27 19:30:18.555567 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.555547    2579 scope.go:117] "RemoveContainer" containerID="285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754"
Jul 27 19:30:18.561988 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.561961    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562428    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfba2a95-8524-4c55-bc1f-9c7cc3abc933" containerName="console"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562447    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfba2a95-8524-4c55-bc1f-9c7cc3abc933" containerName="console"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562459    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy-web"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562467    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy-web"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562482    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="config-reloader"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562491    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="config-reloader"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562505    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" containerName="console"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562513    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" containerName="console"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562525    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="init-config-reloader"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562533    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="init-config-reloader"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562543    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="prom-label-proxy"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562554    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="prom-label-proxy"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562574    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy-metric"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562582    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy-metric"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562601    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="alertmanager"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562609    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="alertmanager"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562624    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy"
Jul 27 19:30:18.562639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562633    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562712    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy-metric"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562727    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5f81b5b-648f-49ea-ae0f-fe3bd82e3f27" containerName="console"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562739    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="prom-label-proxy"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562748    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="alertmanager"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562756    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy-web"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562765    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="kube-rbac-proxy"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562772    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfba2a95-8524-4c55-bc1f-9c7cc3abc933" containerName="console"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.562780    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" containerName="config-reloader"
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564793    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564818    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-tls-assets\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564833    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnwnd\" (UniqueName: \"kubernetes.io/projected/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-kube-api-access-rnwnd\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564847    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564860    2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-cluster-tls-config\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564874    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564886    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-main-tls\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564897    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-out\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564907    2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564919    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564929    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-web-config\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.564955 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.564941    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f68e06-2d39-472c-8c5e-fc15bc9143bb-metrics-client-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:18.566613 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.566593    2579 scope.go:117] "RemoveContainer" containerID="333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb"
Jul 27 19:30:18.568659 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:30:18.568624    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb\": container with ID starting with 333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb not found: ID does not exist" containerID="333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb"
Jul 27 19:30:18.568787 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.568669    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb"} err="failed to get container status \"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb\": rpc error: code = NotFound desc = could not find container \"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb\": container with ID starting with 333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb not found: ID does not exist"
Jul 27 19:30:18.568787 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.568693    2579 scope.go:117] "RemoveContainer" containerID="2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e"
Jul 27 19:30:18.569378 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:30:18.569342    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e\": container with ID starting with 2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e not found: ID does not exist" containerID="2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e"
Jul 27 19:30:18.569378 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.569385    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e"} err="failed to get container status \"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e\": rpc error: code = NotFound desc = could not find container \"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e\": container with ID starting with 2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e not found: ID does not exist"
Jul 27 19:30:18.569565 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.569407    2579 scope.go:117] "RemoveContainer" containerID="6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b"
Jul 27 19:30:18.569750 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:30:18.569728    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b\": container with ID starting with 6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b not found: ID does not exist" containerID="6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b"
Jul 27 19:30:18.569810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.569758    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b"} err="failed to get container status \"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b\": rpc error: code = NotFound desc = could not find container \"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b\": container with ID starting with 6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b not found: ID does not exist"
Jul 27 19:30:18.569810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.569776    2579 scope.go:117] "RemoveContainer" containerID="10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8"
Jul 27 19:30:18.570046 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:30:18.570026    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8\": container with ID starting with 10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8 not found: ID does not exist" containerID="10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8"
Jul 27 19:30:18.570124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.570056    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8"} err="failed to get container status \"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8\": rpc error: code = NotFound desc = could not find container \"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8\": container with ID starting with 10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8 not found: ID does not exist"
Jul 27 19:30:18.570124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.570079    2579 scope.go:117] "RemoveContainer" containerID="882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743"
Jul 27 19:30:18.570373 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:30:18.570354    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743\": container with ID starting with 882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743 not found: ID does not exist" containerID="882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743"
Jul 27 19:30:18.570420 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.570396    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743"} err="failed to get container status \"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743\": rpc error: code = NotFound desc = could not find container \"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743\": container with ID starting with 882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743 not found: ID does not exist"
Jul 27 19:30:18.570420 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.570416    2579 scope.go:117] "RemoveContainer" containerID="9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903"
Jul 27 19:30:18.570641 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:30:18.570624    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903\": container with ID starting with 9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903 not found: ID does not exist" containerID="9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903"
Jul 27 19:30:18.570683 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.570645    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903"} err="failed to get container status \"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903\": rpc error: code = NotFound desc = could not find container \"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903\": container with ID starting with 9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903 not found: ID does not exist"
Jul 27 19:30:18.570683 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.570660    2579 scope.go:117] "RemoveContainer" containerID="285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754"
Jul 27 19:30:18.570902 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:30:18.570886    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754\": container with ID starting with 285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754 not found: ID does not exist" containerID="285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754"
Jul 27 19:30:18.570960 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.570909    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754"} err="failed to get container status \"285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754\": rpc error: code = NotFound desc = could not find container \"285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754\": container with ID starting with 285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754 not found: ID does not exist"
Jul 27 19:30:18.570960 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.570928    2579 scope.go:117] "RemoveContainer" containerID="333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb"
Jul 27 19:30:18.571122 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.571104    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb"} err="failed to get container status \"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb\": rpc error: code = NotFound desc = could not find container \"333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb\": container with ID starting with 333db6973e0c553ae2e5989cabcedc1a181c1c1a8464d8a815314a31dfd166eb not found: ID does not exist"
Jul 27 19:30:18.571165 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.571122    2579 scope.go:117] "RemoveContainer" containerID="2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e"
Jul 27 19:30:18.571405 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.571385    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e"} err="failed to get container status \"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e\": rpc error: code = NotFound desc = could not find container \"2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e\": container with ID starting with 2ba2791e040e32d5881a5a085362e56e0afcd10dc1314a05b24b01eba9c1c73e not found: ID does not exist"
Jul 27 19:30:18.571462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.571406    2579 scope.go:117] "RemoveContainer" containerID="6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b"
Jul 27 19:30:18.571648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.571632    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b"} err="failed to get container status \"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b\": rpc error: code = NotFound desc = could not find container \"6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b\": container with ID starting with 6a9b0cd50d763a1bfdb3885ab68dfca7457745a6f2cf8dd84f398d0f521bcd9b not found: ID does not exist"
Jul 27 19:30:18.571701 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.571649    2579 scope.go:117] "RemoveContainer" containerID="10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8"
Jul 27 19:30:18.571843 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.571827    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8"} err="failed to get container status \"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8\": rpc error: code = NotFound desc = could not find container \"10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8\": container with ID starting with 10db538ecf759aa04b5fc907f38077dc465acfb70206cc240860bde13d1d6df8 not found: ID does not exist"
Jul 27 19:30:18.571889 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.571842    2579 scope.go:117] "RemoveContainer" containerID="882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743"
Jul 27 19:30:18.572065 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.572043    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743"} err="failed to get container status \"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743\": rpc error: code = NotFound desc = could not find container \"882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743\": container with ID starting with 882f83a2eb6b7f4d27515c33738388e03e849b3a7aad5982d184762a6e654743 not found: ID does not exist"
Jul 27 19:30:18.572114 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.572067    2579 scope.go:117] "RemoveContainer" containerID="9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903"
Jul 27 19:30:18.572284 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.572266    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903"} err="failed to get container status \"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903\": rpc error: code = NotFound desc = could not find container \"9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903\": container with ID starting with 9b32d548d51e7a25f13c1f9833c4add0ee0d9728d8db51ab5f4b115db3a5e903 not found: ID does not exist"
Jul 27 19:30:18.572361 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.572286    2579 scope.go:117] "RemoveContainer" containerID="285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754"
Jul 27 19:30:18.572361 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.572289    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.572640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.572536    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754"} err="failed to get container status \"285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754\": rpc error: code = NotFound desc = could not find container \"285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754\": container with ID starting with 285b39c1d5bf6d6ed3e73d65a99e84c6c8e583324b468b1d7ab76b0378558754 not found: ID does not exist"
Jul 27 19:30:18.575410 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.575143    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 27 19:30:18.575410 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.575149    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 27 19:30:18.576363 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.576138    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 27 19:30:18.576363 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.576200    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 27 19:30:18.576811 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.576588    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 27 19:30:18.576811 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.576662    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 27 19:30:18.576958 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.576887    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 27 19:30:18.577013 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.576959    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fd9vk\""
Jul 27 19:30:18.577013 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.576997    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 27 19:30:18.578597 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.578572    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:18.585309 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.584057    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 27 19:30:18.665398 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665581 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665402    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665581 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665441    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665581 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665581 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665581 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-web-config\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665866 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665866 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fnm6\" (UniqueName: \"kubernetes.io/projected/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-kube-api-access-6fnm6\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.665866 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.665799    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-config-out\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766440 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766510    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-web-config\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766534    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766579    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fnm6\" (UniqueName: \"kubernetes.io/projected/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-kube-api-access-6fnm6\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-config-out\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766883 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766662    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766883 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766883 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766883 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766883 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766806    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.766883 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.766840    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.767280 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.767159    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.767593 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.767566    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.768221 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.768199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.769533 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.769509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-config-out\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.769661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.769637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-web-config\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.769737 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.769637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.769847 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.769822    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.769958 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.769844    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.769958 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.769916    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.770052 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.769968    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.770753 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.770738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.771445 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.771427    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.774473 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.774450    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fnm6\" (UniqueName: \"kubernetes.io/projected/d7930482-e1d2-42f5-b2e9-bb5966c50ab7-kube-api-access-6fnm6\") pod \"alertmanager-main-0\" (UID: \"d7930482-e1d2-42f5-b2e9-bb5966c50ab7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:18.888393 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:18.888305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 19:30:19.027096 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:19.027052    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 19:30:19.029090 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:30:19.029061    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7930482_e1d2_42f5_b2e9_bb5966c50ab7.slice/crio-e4e35191e031639232803101ef76250016750ef0056251840ff56990a11dea9a WatchSource:0}: Error finding container e4e35191e031639232803101ef76250016750ef0056251840ff56990a11dea9a: Status 404 returned error can't find the container with id e4e35191e031639232803101ef76250016750ef0056251840ff56990a11dea9a
Jul 27 19:30:19.513801 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:19.513767    2579 generic.go:356] "Generic (PLEG): container finished" podID="d7930482-e1d2-42f5-b2e9-bb5966c50ab7" containerID="91a63f0ae70e33c6f5df8bbd9876c1a0025ea3381335e29595d0b51f127e2446" exitCode=0
Jul 27 19:30:19.514278 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:19.513857    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7930482-e1d2-42f5-b2e9-bb5966c50ab7","Type":"ContainerDied","Data":"91a63f0ae70e33c6f5df8bbd9876c1a0025ea3381335e29595d0b51f127e2446"}
Jul 27 19:30:19.514278 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:19.513937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7930482-e1d2-42f5-b2e9-bb5966c50ab7","Type":"ContainerStarted","Data":"e4e35191e031639232803101ef76250016750ef0056251840ff56990a11dea9a"}
Jul 27 19:30:20.487772 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:20.487740    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3f68e06-2d39-472c-8c5e-fc15bc9143bb" path="/var/lib/kubelet/pods/d3f68e06-2d39-472c-8c5e-fc15bc9143bb/volumes"
Jul 27 19:30:20.521363 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:20.521335    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7930482-e1d2-42f5-b2e9-bb5966c50ab7","Type":"ContainerStarted","Data":"3c823ab88532a031de856ae8062b535ac3821b19f5ed81f348006671f5faca2a"}
Jul 27 19:30:20.521744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:20.521398    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7930482-e1d2-42f5-b2e9-bb5966c50ab7","Type":"ContainerStarted","Data":"55172c4884bac03e7f6597c8627426ca043dcbdbadd67667e9fbbaf63cdf01af"}
Jul 27 19:30:20.521744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:20.521425    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7930482-e1d2-42f5-b2e9-bb5966c50ab7","Type":"ContainerStarted","Data":"f3873d5759f2e674dd7757bcd2aab1d34bea4a5647cf64133756a73a228f2a82"}
Jul 27 19:30:20.521744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:20.521450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7930482-e1d2-42f5-b2e9-bb5966c50ab7","Type":"ContainerStarted","Data":"418b1d36595b23aeb190200606a878f0492ec7d2e3ff4621c7071b177f1ca4b9"}
Jul 27 19:30:20.521744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:20.521479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7930482-e1d2-42f5-b2e9-bb5966c50ab7","Type":"ContainerStarted","Data":"2d0245171e25551ea875ce52f289d73969fe28a2d0904a856c67fe9d7a36486d"}
Jul 27 19:30:20.521744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:20.521503    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7930482-e1d2-42f5-b2e9-bb5966c50ab7","Type":"ContainerStarted","Data":"73e6157bb939b5c30edb414fa49292f1fff4ba60f8569cffa3309aa0909a0371"}
Jul 27 19:30:20.548918 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:20.548868    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.532907496 podStartE2EDuration="2.548847123s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.939627ms" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:20.547648066 +0000 UTC m=+272.638999564" watchObservedRunningTime="2026-07-27 19:30:20.548847123 +0000 UTC m=+272.640198619"
Jul 27 19:30:21.293574 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.292917    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:30:21.293770 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.293594    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="prometheus" containerID="cri-o://624f6db288f3d6988364e20c90f9450ce132b10844212cc197f71d3c07f29a76" gracePeriod=600
Jul 27 19:30:21.293770 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.293625    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy-web" containerID="cri-o://e866943e85aa7598d9f857d3534a7714b79fc30907a1c79fd82204c7ac329543" gracePeriod=600
Jul 27 19:30:21.293916 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.293794    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy" containerID="cri-o://b97af28f7fdae0c1dc7d577496fd77b991a1757a7f6e8f4512c79bf875db31e2" gracePeriod=600
Jul 27 19:30:21.293916 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.293804    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="thanos-sidecar" containerID="cri-o://56781ce410983caf39c23adcc9793a551616c88bbee646905305b80a793aaf3b" gracePeriod=600
Jul 27 19:30:21.293916 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.293905    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="config-reloader" containerID="cri-o://3103a2568515159f5e8ea36624fbb84f19689f0842679ccf0a08b336b7c2ee2d" gracePeriod=600
Jul 27 19:30:21.297150 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.294591    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy-thanos" containerID="cri-o://107c6beec2a1e1a8de6542fee491ec91638679457bf5f25c20cd779ab3ad8b03" gracePeriod=600
Jul 27 19:30:21.532876 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.532839    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerID="107c6beec2a1e1a8de6542fee491ec91638679457bf5f25c20cd779ab3ad8b03" exitCode=0
Jul 27 19:30:21.532876 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.532872    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerID="b97af28f7fdae0c1dc7d577496fd77b991a1757a7f6e8f4512c79bf875db31e2" exitCode=0
Jul 27 19:30:21.532876 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.532881    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerID="e866943e85aa7598d9f857d3534a7714b79fc30907a1c79fd82204c7ac329543" exitCode=0
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.532892    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerID="56781ce410983caf39c23adcc9793a551616c88bbee646905305b80a793aaf3b" exitCode=0
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.532900    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerID="3103a2568515159f5e8ea36624fbb84f19689f0842679ccf0a08b336b7c2ee2d" exitCode=0
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.532908    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerID="624f6db288f3d6988364e20c90f9450ce132b10844212cc197f71d3c07f29a76" exitCode=0
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.532908    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerDied","Data":"107c6beec2a1e1a8de6542fee491ec91638679457bf5f25c20cd779ab3ad8b03"}
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.532995    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerDied","Data":"b97af28f7fdae0c1dc7d577496fd77b991a1757a7f6e8f4512c79bf875db31e2"}
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.533042    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerDied","Data":"e866943e85aa7598d9f857d3534a7714b79fc30907a1c79fd82204c7ac329543"}
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.533080    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerDied","Data":"56781ce410983caf39c23adcc9793a551616c88bbee646905305b80a793aaf3b"}
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.533118    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerDied","Data":"3103a2568515159f5e8ea36624fbb84f19689f0842679ccf0a08b336b7c2ee2d"}
Jul 27 19:30:21.533433 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.533157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerDied","Data":"624f6db288f3d6988364e20c90f9450ce132b10844212cc197f71d3c07f29a76"}
Jul 27 19:30:21.551452 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.551382    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:21.699967 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.699927    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-0\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.699967 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.699971    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700000    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-db\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700016    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-kube-rbac-proxy\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700033    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-kubelet-serving-ca-bundle\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-metrics-client-certs\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700074    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-web-config\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700092    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-trusted-ca-bundle\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700112    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brgrv\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-kube-api-access-brgrv\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700147    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-grpc-tls\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700171    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-2\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700216    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-tls-assets\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700272    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-thanos-prometheus-http-client-file\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700300    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-1\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700341    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-metrics-client-ca\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700381    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700425    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config-out\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700436    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700451    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-tls\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700514    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700549    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-serving-certs-ca-bundle\") pod \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\" (UID: \"1f52bcd5-25d1-450d-ac64-68cdc5c46262\") "
Jul 27 19:30:21.700890 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.700845    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.701401 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.701176    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:21.701401 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.701173    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:21.701584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.701556    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:21.701897 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.701870    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:21.702119 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.702094    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:21.703014 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.702986    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:21.703387 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.703360    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:21.704235 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.704199    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config" (OuterVolumeSpecName: "config") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.704350 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.704294    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-kube-api-access-brgrv" (OuterVolumeSpecName: "kube-api-access-brgrv") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "kube-api-access-brgrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:21.704411 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.704384    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.704700 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.704674    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:21.704783 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.704730    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.704844 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.704826    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.705214 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.705172    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.705358 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.705319    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.705489 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.705468    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.705534 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.705514    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config-out" (OuterVolumeSpecName: "config-out") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:21.706146 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.706127    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.715684 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.715656    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-web-config" (OuterVolumeSpecName: "web-config") pod "1f52bcd5-25d1-450d-ac64-68cdc5c46262" (UID: "1f52bcd5-25d1-450d-ac64-68cdc5c46262"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:21.801370 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801340    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801370 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801368    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801381    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-db\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801392    2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-kube-rbac-proxy\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801402    2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-metrics-client-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801414    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-web-config\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801427    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801440    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brgrv\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-kube-api-access-brgrv\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801450    2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-grpc-tls\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801462    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801474    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f52bcd5-25d1-450d-ac64-68cdc5c46262-tls-assets\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801483    2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801491    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801501    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-metrics-client-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801509    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801518    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config-out\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801525    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801532    2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f52bcd5-25d1-450d-ac64-68cdc5c46262-config\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:21.801572 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:21.801542    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f52bcd5-25d1-450d-ac64-68cdc5c46262-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:30:22.540769 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.540741    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.541191 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.540739    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f52bcd5-25d1-450d-ac64-68cdc5c46262","Type":"ContainerDied","Data":"2ebc1b4d28219c3f559929f7c82fd4be5557509c3c2dad603c09a816dbf5144a"}
Jul 27 19:30:22.541191 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.540893    2579 scope.go:117] "RemoveContainer" containerID="107c6beec2a1e1a8de6542fee491ec91638679457bf5f25c20cd779ab3ad8b03"
Jul 27 19:30:22.549051 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.549033    2579 scope.go:117] "RemoveContainer" containerID="b97af28f7fdae0c1dc7d577496fd77b991a1757a7f6e8f4512c79bf875db31e2"
Jul 27 19:30:22.555930 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.555911    2579 scope.go:117] "RemoveContainer" containerID="e866943e85aa7598d9f857d3534a7714b79fc30907a1c79fd82204c7ac329543"
Jul 27 19:30:22.562787 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.562718    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:30:22.563812 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.563790    2579 scope.go:117] "RemoveContainer" containerID="56781ce410983caf39c23adcc9793a551616c88bbee646905305b80a793aaf3b"
Jul 27 19:30:22.566120 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.566102    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:30:22.571516 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.571497    2579 scope.go:117] "RemoveContainer" containerID="3103a2568515159f5e8ea36624fbb84f19689f0842679ccf0a08b336b7c2ee2d"
Jul 27 19:30:22.578219 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.578199    2579 scope.go:117] "RemoveContainer" containerID="624f6db288f3d6988364e20c90f9450ce132b10844212cc197f71d3c07f29a76"
Jul 27 19:30:22.585553 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.585517    2579 scope.go:117] "RemoveContainer" containerID="5fb03e878bd3ca609ccf499894dab28a5a53adb413238bafc9415a5e8b6f044b"
Jul 27 19:30:22.597200 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597172    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:30:22.597721 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597695    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="init-config-reloader"
Jul 27 19:30:22.597721 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597723    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="init-config-reloader"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597737    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="config-reloader"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597746    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="config-reloader"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597763    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="prometheus"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597771    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="prometheus"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597784    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597792    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597807    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy-web"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597815    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy-web"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597829    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="thanos-sidecar"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597838    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="thanos-sidecar"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597848    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy-thanos"
Jul 27 19:30:22.597863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597856    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy-thanos"
Jul 27 19:30:22.598453 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597934    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy-web"
Jul 27 19:30:22.598453 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597946    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy"
Jul 27 19:30:22.598453 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597957    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="kube-rbac-proxy-thanos"
Jul 27 19:30:22.598453 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597970    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="config-reloader"
Jul 27 19:30:22.598453 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597980    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="prometheus"
Jul 27 19:30:22.598453 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.597988    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" containerName="thanos-sidecar"
Jul 27 19:30:22.603922 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.603902    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.606655 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.606630    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 27 19:30:22.606655 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.606648    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 27 19:30:22.606827 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.606626    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 27 19:30:22.607098 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607075    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 27 19:30:22.607230 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607147    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 27 19:30:22.607230 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607148    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 27 19:30:22.607423 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607192    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 27 19:30:22.607423 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607224    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fkckc9e9fot2m\""
Jul 27 19:30:22.607521 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607502    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 27 19:30:22.607582 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607544    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 27 19:30:22.607582 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607558    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 27 19:30:22.607679 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607626    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 27 19:30:22.607995 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.607980    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 27 19:30:22.608571 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.608550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nh2bt\""
Jul 27 19:30:22.610127 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.610106    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 27 19:30:22.612049 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.611941    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 27 19:30:22.620373 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.620348    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:30:22.709289 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5fj5\" (UniqueName: \"kubernetes.io/projected/c01d3c88-06e2-4885-a3c6-06cb52f931df-kube-api-access-q5fj5\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709436    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709472 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-config\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709637 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709637 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709492    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c01d3c88-06e2-4885-a3c6-06cb52f931df-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709637 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709637 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709637 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709637 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c01d3c88-06e2-4885-a3c6-06cb52f931df-config-out\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709637 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709637    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-web-config\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709721    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.709851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.709791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.810462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.810427    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.810462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.810465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.810667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.810485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.810667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.810520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.810667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.810552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.810667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.810576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.810667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.810600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5fj5\" (UniqueName: \"kubernetes.io/projected/c01d3c88-06e2-4885-a3c6-06cb52f931df-kube-api-access-q5fj5\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.810667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.810630    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811337 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-config\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811477 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811477 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c01d3c88-06e2-4885-a3c6-06cb52f931df-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811477 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811477 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811477 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811518    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c01d3c88-06e2-4885-a3c6-06cb52f931df-config-out\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-web-config\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.811751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.811676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.812365 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.812337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.814605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.813682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.814605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.813682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.814605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.813711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.814605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.813751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.814605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.814029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.814605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.814311    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.814605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.814383    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.814605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.814600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.815086 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.814975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.815376 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.815355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-config\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.815449 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.815399    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.816124 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.816101    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.816215 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.816168    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c01d3c88-06e2-4885-a3c6-06cb52f931df-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.816333 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.816300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c01d3c88-06e2-4885-a3c6-06cb52f931df-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.816517 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.816497    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c01d3c88-06e2-4885-a3c6-06cb52f931df-config-out\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.816813 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.816789    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-web-config\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.816941 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.816920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c01d3c88-06e2-4885-a3c6-06cb52f931df-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.818584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.818561    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5fj5\" (UniqueName: \"kubernetes.io/projected/c01d3c88-06e2-4885-a3c6-06cb52f931df-kube-api-access-q5fj5\") pod \"prometheus-k8s-0\" (UID: \"c01d3c88-06e2-4885-a3c6-06cb52f931df\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:22.920466 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:22.920435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:23.067667 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:23.067587    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 19:30:23.070593 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:30:23.070567    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc01d3c88_06e2_4885_a3c6_06cb52f931df.slice/crio-95aa6dca1b9f42f76523c76473766b021bc846d5ca30b06d9f5bdaead5f28fa7 WatchSource:0}: Error finding container 95aa6dca1b9f42f76523c76473766b021bc846d5ca30b06d9f5bdaead5f28fa7: Status 404 returned error can't find the container with id 95aa6dca1b9f42f76523c76473766b021bc846d5ca30b06d9f5bdaead5f28fa7
Jul 27 19:30:23.546332 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:23.546301    2579 generic.go:356] "Generic (PLEG): container finished" podID="c01d3c88-06e2-4885-a3c6-06cb52f931df" containerID="43c99e584d0c3af7b6cbc2b6bd08aff9a22aa0a310cff2cc7915d103d9364196" exitCode=0
Jul 27 19:30:23.546731 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:23.546343    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c01d3c88-06e2-4885-a3c6-06cb52f931df","Type":"ContainerDied","Data":"43c99e584d0c3af7b6cbc2b6bd08aff9a22aa0a310cff2cc7915d103d9364196"}
Jul 27 19:30:23.546731 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:23.546393    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c01d3c88-06e2-4885-a3c6-06cb52f931df","Type":"ContainerStarted","Data":"95aa6dca1b9f42f76523c76473766b021bc846d5ca30b06d9f5bdaead5f28fa7"}
Jul 27 19:30:24.490994 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:24.490959    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f52bcd5-25d1-450d-ac64-68cdc5c46262" path="/var/lib/kubelet/pods/1f52bcd5-25d1-450d-ac64-68cdc5c46262/volumes"
Jul 27 19:30:24.553382 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:24.553341    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c01d3c88-06e2-4885-a3c6-06cb52f931df","Type":"ContainerStarted","Data":"e72aaf7e3cfbb4ba89c81282f322effdd22b377d10e4a71532c484445b36a002"}
Jul 27 19:30:24.553758 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:24.553429    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c01d3c88-06e2-4885-a3c6-06cb52f931df","Type":"ContainerStarted","Data":"866f9128bf47b91dac1e2c6591ecf964705c8fe00d75280fe43d409d87bc0dab"}
Jul 27 19:30:24.553758 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:24.553466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c01d3c88-06e2-4885-a3c6-06cb52f931df","Type":"ContainerStarted","Data":"c8cc77f96f10c6dcb8fb2ad88b4a79d2e145d267e01041b876a7f91502f1e5cc"}
Jul 27 19:30:24.553758 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:24.553502    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c01d3c88-06e2-4885-a3c6-06cb52f931df","Type":"ContainerStarted","Data":"82d11eae42d7289aaf39800a9fd751cddcb6b9113b934b2d7e9b025ea781f6aa"}
Jul 27 19:30:24.553758 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:24.553540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c01d3c88-06e2-4885-a3c6-06cb52f931df","Type":"ContainerStarted","Data":"33dc5a45bd71bc31add2823698c404495c3a6a8f4cb5a37091af28d56ac0b7a4"}
Jul 27 19:30:24.553758 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:24.553573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c01d3c88-06e2-4885-a3c6-06cb52f931df","Type":"ContainerStarted","Data":"1b972a258adb67ed24c1d7fdbfb018a6a7e3bb10e013789eb3a400105a8dd758"}
Jul 27 19:30:24.585359 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:24.585300    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.566076043 podStartE2EDuration="2.585279695s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.203652ms" isStatefulPod=false podCreationTimestamp="2026-07-27 19:30:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:24.583171435 +0000 UTC m=+276.674522974" watchObservedRunningTime="2026-07-27 19:30:24.585279695 +0000 UTC m=+276.676631193"
Jul 27 19:30:26.730956 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:26.730917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:26.793959 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:26.793921    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:27.921456 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:27.921421    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:30:36.453351 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:36.453316    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9km82"
Jul 27 19:30:37.748194 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:37.748155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:37.849276 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:37.849229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:38.032997 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:38.032909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:38.088027 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:38.087989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:30:48.293805 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:48.293768    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:30:48.295282 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:48.295233    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:30:48.300125 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:48.300101    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:30:48.301362 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:48.301344    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:30:48.301877 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:30:48.301849    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 19:31:03.258380 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:03.258339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:05.181359 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:05.181320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:05.239053 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:05.239019    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:22.921549 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:22.921503    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:31:22.937167 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:22.937142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:31:23.794770 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:23.794738    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 19:31:31.171127 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:31.171081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:32.941058 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:32.941028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:38.888783 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:38.888740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:38.948778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:38.948740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:39.033739 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:39.033704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:39.087559 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:39.087513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:39.692633 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:39.692557    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:39.795159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:39.795124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:43.144379 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:43.144345    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:43.210198 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:43.210171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:49.971204 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:49.971166    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:50.072004 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:50.071970    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:50.141038 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:50.141007    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:55.643378 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:55.643344    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:31:55.745817 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:31:55.745779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:30.921877 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:30.921845    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:31.021865 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:31.021833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:41.359939 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:41.359907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:52.616943 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:52.616905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:53.594515 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:53.594472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:56.227860 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:56.227824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:56.328286 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:56.328252    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:57.828862 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:57.828832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:57.928520 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:57.928485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:58.443466 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:58.443431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:32:58.572025 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:32:58.571989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:00.236400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:00.236363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:00.644661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:00.644571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:03.229183 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:03.229144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:03.288437 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:03.288398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:07.188365 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:07.188327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:07.287336 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:07.287297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:07.641017 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:07.640930    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:08.186344 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.186313    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7v5b6"]
Jul 27 19:33:08.188587 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.188568    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7v5b6"
Jul 27 19:33:08.191176 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.191157    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 19:33:08.191369 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.191157    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 19:33:08.191369 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.191294    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-m6lgn\""
Jul 27 19:33:08.198096 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.198075    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7v5b6"]
Jul 27 19:33:08.325156 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.325108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba0a16b0-8412-462b-ba86-9c93ee3247b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-7v5b6\" (UID: \"ba0a16b0-8412-462b-ba86-9c93ee3247b5\") " pod="cert-manager/cert-manager-55b4d954d5-7v5b6"
Jul 27 19:33:08.325374 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.325182    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pmp4\" (UniqueName: \"kubernetes.io/projected/ba0a16b0-8412-462b-ba86-9c93ee3247b5-kube-api-access-7pmp4\") pod \"cert-manager-55b4d954d5-7v5b6\" (UID: \"ba0a16b0-8412-462b-ba86-9c93ee3247b5\") " pod="cert-manager/cert-manager-55b4d954d5-7v5b6"
Jul 27 19:33:08.426167 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.426127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba0a16b0-8412-462b-ba86-9c93ee3247b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-7v5b6\" (UID: \"ba0a16b0-8412-462b-ba86-9c93ee3247b5\") " pod="cert-manager/cert-manager-55b4d954d5-7v5b6"
Jul 27 19:33:08.426409 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.426193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pmp4\" (UniqueName: \"kubernetes.io/projected/ba0a16b0-8412-462b-ba86-9c93ee3247b5-kube-api-access-7pmp4\") pod \"cert-manager-55b4d954d5-7v5b6\" (UID: \"ba0a16b0-8412-462b-ba86-9c93ee3247b5\") " pod="cert-manager/cert-manager-55b4d954d5-7v5b6"
Jul 27 19:33:08.434054 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.434024    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba0a16b0-8412-462b-ba86-9c93ee3247b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-7v5b6\" (UID: \"ba0a16b0-8412-462b-ba86-9c93ee3247b5\") " pod="cert-manager/cert-manager-55b4d954d5-7v5b6"
Jul 27 19:33:08.434218 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.434200    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pmp4\" (UniqueName: \"kubernetes.io/projected/ba0a16b0-8412-462b-ba86-9c93ee3247b5-kube-api-access-7pmp4\") pod \"cert-manager-55b4d954d5-7v5b6\" (UID: \"ba0a16b0-8412-462b-ba86-9c93ee3247b5\") " pod="cert-manager/cert-manager-55b4d954d5-7v5b6"
Jul 27 19:33:08.507521 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.507430    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7v5b6"
Jul 27 19:33:08.634853 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.634781    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7v5b6"]
Jul 27 19:33:08.637290 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:33:08.637262    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba0a16b0_8412_462b_ba86_9c93ee3247b5.slice/crio-e10b1568f158e161b0243821d31b542e91a76c96787f83a0bc4c19814a4c43b2 WatchSource:0}: Error finding container e10b1568f158e161b0243821d31b542e91a76c96787f83a0bc4c19814a4c43b2: Status 404 returned error can't find the container with id e10b1568f158e161b0243821d31b542e91a76c96787f83a0bc4c19814a4c43b2
Jul 27 19:33:08.639765 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:08.639749    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:33:09.199604 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:09.199566    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7v5b6" event={"ID":"ba0a16b0-8412-462b-ba86-9c93ee3247b5","Type":"ContainerStarted","Data":"e10b1568f158e161b0243821d31b542e91a76c96787f83a0bc4c19814a4c43b2"}
Jul 27 19:33:09.628920 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:09.628709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:09.726190 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:09.726068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:11.687220 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.687197    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"]
Jul 27 19:33:11.689480 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.689462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"
Jul 27 19:33:11.692104 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.692084    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-sp88x\""
Jul 27 19:33:11.692209 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.692084    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 19:33:11.692209 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.692092    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:33:11.699069 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.699046    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"]
Jul 27 19:33:11.760066 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.760031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b0ad86e-b6ca-488c-bbfb-6528f33245fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hkhh4\" (UID: \"1b0ad86e-b6ca-488c-bbfb-6528f33245fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"
Jul 27 19:33:11.760268 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.760082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9qpd\" (UniqueName: \"kubernetes.io/projected/1b0ad86e-b6ca-488c-bbfb-6528f33245fc-kube-api-access-r9qpd\") pod \"openshift-lws-operator-fd8ccff4c-hkhh4\" (UID: \"1b0ad86e-b6ca-488c-bbfb-6528f33245fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"
Jul 27 19:33:11.860789 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.860752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9qpd\" (UniqueName: \"kubernetes.io/projected/1b0ad86e-b6ca-488c-bbfb-6528f33245fc-kube-api-access-r9qpd\") pod \"openshift-lws-operator-fd8ccff4c-hkhh4\" (UID: \"1b0ad86e-b6ca-488c-bbfb-6528f33245fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"
Jul 27 19:33:11.860984 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.860906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b0ad86e-b6ca-488c-bbfb-6528f33245fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hkhh4\" (UID: \"1b0ad86e-b6ca-488c-bbfb-6528f33245fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"
Jul 27 19:33:11.861281 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.861233    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b0ad86e-b6ca-488c-bbfb-6528f33245fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hkhh4\" (UID: \"1b0ad86e-b6ca-488c-bbfb-6528f33245fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"
Jul 27 19:33:11.869327 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:11.869301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9qpd\" (UniqueName: \"kubernetes.io/projected/1b0ad86e-b6ca-488c-bbfb-6528f33245fc-kube-api-access-r9qpd\") pod \"openshift-lws-operator-fd8ccff4c-hkhh4\" (UID: \"1b0ad86e-b6ca-488c-bbfb-6528f33245fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"
Jul 27 19:33:12.010210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:12.010180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"
Jul 27 19:33:12.136115 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:12.136081    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4"]
Jul 27 19:33:12.138937 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:33:12.138913    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b0ad86e_b6ca_488c_bbfb_6528f33245fc.slice/crio-b1af0f2572127ae33567b691ee95c90661b8a69eb90b44756cc51c5d117b50a2 WatchSource:0}: Error finding container b1af0f2572127ae33567b691ee95c90661b8a69eb90b44756cc51c5d117b50a2: Status 404 returned error can't find the container with id b1af0f2572127ae33567b691ee95c90661b8a69eb90b44756cc51c5d117b50a2
Jul 27 19:33:12.214224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:12.214185    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7v5b6" event={"ID":"ba0a16b0-8412-462b-ba86-9c93ee3247b5","Type":"ContainerStarted","Data":"cad8e643135fb546a6e9653bc938a83e12f0d3387904b33535af22217fb95c3f"}
Jul 27 19:33:12.215399 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:12.215376    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4" event={"ID":"1b0ad86e-b6ca-488c-bbfb-6528f33245fc","Type":"ContainerStarted","Data":"b1af0f2572127ae33567b691ee95c90661b8a69eb90b44756cc51c5d117b50a2"}
Jul 27 19:33:12.230263 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:12.230207    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-7v5b6" podStartSLOduration=1.215592799 podStartE2EDuration="4.230192562s" totalImagesPullingTime="3.014599763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:12.228467624 +0000 UTC m=+444.319819124" watchObservedRunningTime="2026-07-27 19:33:12.230192562 +0000 UTC m=+444.321544060"
Jul 27 19:33:15.233136 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:15.233102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4" event={"ID":"1b0ad86e-b6ca-488c-bbfb-6528f33245fc","Type":"ContainerStarted","Data":"9714f0383a449f2827790f01dc197594754aad278284b75e3b75bf26f21357f2"}
Jul 27 19:33:15.252442 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:15.252387    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hkhh4" podStartSLOduration=1.5916936860000002 podStartE2EDuration="4.252367164s" totalImagesPullingTime="2.660673478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:15.250549072 +0000 UTC m=+447.341900572" watchObservedRunningTime="2026-07-27 19:33:15.252367164 +0000 UTC m=+447.343718665"
Jul 27 19:33:15.961483 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:15.961443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:16.062102 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:16.062063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:17.580764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:17.580729    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:30.632205 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:30.632171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:30.728681 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:30.728646    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:31.236877 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.236843    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:31.473596 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.473558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:31.572454 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.572360    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:31.611489 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.611456    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"]
Jul 27 19:33:31.618710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.618682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.622662 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.622636    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lhqz6\""
Jul 27 19:33:31.622875 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.622849    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 19:33:31.623033 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.622958    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 19:33:31.623312 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.623294    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 19:33:31.623789 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.623775    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 19:33:31.630627 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.630598    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"]
Jul 27 19:33:31.726333 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.726297    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0fffa1c2-3491-46af-8e71-3ba88f3ca103-manifests\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.726728 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.726341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fffa1c2-3491-46af-8e71-3ba88f3ca103-webhook-cert\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.726728 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.726426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xvgl\" (UniqueName: \"kubernetes.io/projected/0fffa1c2-3491-46af-8e71-3ba88f3ca103-kube-api-access-8xvgl\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.726728 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.726475    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0fffa1c2-3491-46af-8e71-3ba88f3ca103-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.726728 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.726526    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fffa1c2-3491-46af-8e71-3ba88f3ca103-tmp\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.754531 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.754493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:31.827784 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.827685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xvgl\" (UniqueName: \"kubernetes.io/projected/0fffa1c2-3491-46af-8e71-3ba88f3ca103-kube-api-access-8xvgl\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.827784 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.827748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0fffa1c2-3491-46af-8e71-3ba88f3ca103-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.827784 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.827783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fffa1c2-3491-46af-8e71-3ba88f3ca103-tmp\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.828059 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.827822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0fffa1c2-3491-46af-8e71-3ba88f3ca103-manifests\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.828059 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.827851    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fffa1c2-3491-46af-8e71-3ba88f3ca103-webhook-cert\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.828366 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.828334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0fffa1c2-3491-46af-8e71-3ba88f3ca103-manifests\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.830148 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.830120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fffa1c2-3491-46af-8e71-3ba88f3ca103-tmp\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.830322 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.830305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0fffa1c2-3491-46af-8e71-3ba88f3ca103-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.830372 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.830329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fffa1c2-3491-46af-8e71-3ba88f3ca103-webhook-cert\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.835860 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.835839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xvgl\" (UniqueName: \"kubernetes.io/projected/0fffa1c2-3491-46af-8e71-3ba88f3ca103-kube-api-access-8xvgl\") pod \"opendatahub-operator-controller-manager-787b9f94c8-6xt87\" (UID: \"0fffa1c2-3491-46af-8e71-3ba88f3ca103\") " pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.849211 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.849180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:31.931294 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.931238    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:31.953282 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:31.953218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:32.053307 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:32.053273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:32.070803 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:32.070770    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"]
Jul 27 19:33:32.074681 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:33:32.074657    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fffa1c2_3491_46af_8e71_3ba88f3ca103.slice/crio-23ac4a4243ca9b8bcd18bfaaf31b437ffaeaa04b02566ca20dc90633cf9d54b4 WatchSource:0}: Error finding container 23ac4a4243ca9b8bcd18bfaaf31b437ffaeaa04b02566ca20dc90633cf9d54b4: Status 404 returned error can't find the container with id 23ac4a4243ca9b8bcd18bfaaf31b437ffaeaa04b02566ca20dc90633cf9d54b4
Jul 27 19:33:32.150675 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:32.150630    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:32.250262 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:32.250214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:32.300999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:32.300965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87" event={"ID":"0fffa1c2-3491-46af-8e71-3ba88f3ca103","Type":"ContainerStarted","Data":"23ac4a4243ca9b8bcd18bfaaf31b437ffaeaa04b02566ca20dc90633cf9d54b4"}
Jul 27 19:33:32.350286 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:32.350157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:36.322207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:36.322161    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87" event={"ID":"0fffa1c2-3491-46af-8e71-3ba88f3ca103","Type":"ContainerStarted","Data":"7ed930b90a144800feab797834075209417d855a91f22f1a0f24810996a4c954"}
Jul 27 19:33:37.328209 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:37.328128    2579 generic.go:356] "Generic (PLEG): container finished" podID="0fffa1c2-3491-46af-8e71-3ba88f3ca103" containerID="7ed930b90a144800feab797834075209417d855a91f22f1a0f24810996a4c954" exitCode=0
Jul 27 19:33:37.328209 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:37.328175    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87" event={"ID":"0fffa1c2-3491-46af-8e71-3ba88f3ca103","Type":"ContainerDied","Data":"7ed930b90a144800feab797834075209417d855a91f22f1a0f24810996a4c954"}
Jul 27 19:33:38.334614 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:38.334579    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87" event={"ID":"0fffa1c2-3491-46af-8e71-3ba88f3ca103","Type":"ContainerStarted","Data":"d53919c0bdade9ec9417d4108c3dfdd7e581e6493ff8d602cfac7a213f96b9fe"}
Jul 27 19:33:38.335070 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:38.334731    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:38.362064 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:38.362012    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87" podStartSLOduration=2.275263406 podStartE2EDuration="7.361992402s" totalImagesPullingTime="4.05509761s" totalInitContainerRuntime="1.031631386s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:38.360459102 +0000 UTC m=+470.451810600" watchObservedRunningTime="2026-07-27 19:33:38.361992402 +0000 UTC m=+470.453343900"
Jul 27 19:33:42.776399 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:42.776368    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:42.876621 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:42.876584    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:45.957289 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:45.957232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:46.073112 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:46.073079    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:49.341173 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:49.341139    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-787b9f94c8-6xt87"
Jul 27 19:33:50.083827 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.083792    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"]
Jul 27 19:33:50.092357 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.092332    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.097403 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.097379    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zvgdk\""
Jul 27 19:33:50.098113 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.098090    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 19:33:50.098787 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.098765    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 19:33:50.098916 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.098809    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 19:33:50.100557 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.100535    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"]
Jul 27 19:33:50.184627 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.184585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-cert\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.184810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.184635    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-manager-config\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.184810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.184733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-metrics-cert\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.184810 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.184761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ssth\" (UniqueName: \"kubernetes.io/projected/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-kube-api-access-7ssth\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.285815 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.285782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-metrics-cert\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.286033 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.285825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ssth\" (UniqueName: \"kubernetes.io/projected/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-kube-api-access-7ssth\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.286033 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.285849    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-cert\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.286033 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.285901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-manager-config\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.286648 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.286623    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-manager-config\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.288373 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.288342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-metrics-cert\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.288521 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.288504    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-cert\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.294229 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.294198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ssth\" (UniqueName: \"kubernetes.io/projected/12c0ea9a-c39e-4a76-91bc-74b7de3b43e9-kube-api-access-7ssth\") pod \"lws-controller-manager-8cbf9cd8b-t4dv2\" (UID: \"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9\") " pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.402530 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.402431    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:50.553579 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.553557    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"]
Jul 27 19:33:50.556088 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:33:50.556060    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c0ea9a_c39e_4a76_91bc_74b7de3b43e9.slice/crio-01fe1e007cc9dc880b2ff86a781f2cc8f4724d2576e526e6f15e61865af63be6 WatchSource:0}: Error finding container 01fe1e007cc9dc880b2ff86a781f2cc8f4724d2576e526e6f15e61865af63be6: Status 404 returned error can't find the container with id 01fe1e007cc9dc880b2ff86a781f2cc8f4724d2576e526e6f15e61865af63be6
Jul 27 19:33:50.890737 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:50.890707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:51.002288 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:51.002224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:51.404815 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:51.404785    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2" event={"ID":"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9","Type":"ContainerStarted","Data":"01fe1e007cc9dc880b2ff86a781f2cc8f4724d2576e526e6f15e61865af63be6"}
Jul 27 19:33:51.605360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:51.605274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:51.731424 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:51.731363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:51.863812 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:51.863785    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:52.134923 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:52.134824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:52.411050 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:52.410958    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2" event={"ID":"12c0ea9a-c39e-4a76-91bc-74b7de3b43e9","Type":"ContainerStarted","Data":"f2097914a1b33affbe45a8b6bb8cbb735f0e7bb35ba01c451a3ad327d7297161"}
Jul 27 19:33:52.411050 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:52.411028    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:33:52.415388 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:52.415363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:52.429704 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:52.429653    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2" podStartSLOduration=1.183669456 podStartE2EDuration="2.429638402s" totalImagesPullingTime="1.245968946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:52.428381732 +0000 UTC m=+484.519733232" watchObservedRunningTime="2026-07-27 19:33:52.429638402 +0000 UTC m=+484.520989900"
Jul 27 19:33:56.880208 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.880165    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:56.942905 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.942873    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"]
Jul 27 19:33:56.946499 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.946475    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:56.948744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.948721    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 19:33:56.949023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.949007    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hhv8b\""
Jul 27 19:33:56.949828 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.949809    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 19:33:56.949938 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.949811    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 19:33:56.950007 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.949967    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 19:33:56.958808 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:56.958789    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"]
Jul 27 19:33:57.047305 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.047268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbht7\" (UniqueName: \"kubernetes.io/projected/029948f5-1e42-453e-8016-5e11292fe3c7-kube-api-access-vbht7\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.047473 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.047379    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029948f5-1e42-453e-8016-5e11292fe3c7-tmp\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.047473 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.047412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029948f5-1e42-453e-8016-5e11292fe3c7-tls-certs\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.148649 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.148561    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029948f5-1e42-453e-8016-5e11292fe3c7-tmp\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.148649 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.148597    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029948f5-1e42-453e-8016-5e11292fe3c7-tls-certs\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.148649 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.148643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbht7\" (UniqueName: \"kubernetes.io/projected/029948f5-1e42-453e-8016-5e11292fe3c7-kube-api-access-vbht7\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.150851 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.150827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029948f5-1e42-453e-8016-5e11292fe3c7-tmp\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.151106 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.151086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029948f5-1e42-453e-8016-5e11292fe3c7-tls-certs\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.156800 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.156778    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbht7\" (UniqueName: \"kubernetes.io/projected/029948f5-1e42-453e-8016-5e11292fe3c7-kube-api-access-vbht7\") pod \"kube-auth-proxy-766b79c75b-jt2nv\" (UID: \"029948f5-1e42-453e-8016-5e11292fe3c7\") " pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.158568 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.158541    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:57.256105 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.256070    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"
Jul 27 19:33:57.393463 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:33:57.393421    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029948f5_1e42_453e_8016_5e11292fe3c7.slice/crio-58526df43214f13bf37f3f42b09f5306d677fefb337a05b6506cb50325f800b6 WatchSource:0}: Error finding container 58526df43214f13bf37f3f42b09f5306d677fefb337a05b6506cb50325f800b6: Status 404 returned error can't find the container with id 58526df43214f13bf37f3f42b09f5306d677fefb337a05b6506cb50325f800b6
Jul 27 19:33:57.393766 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.393744    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv"]
Jul 27 19:33:57.434559 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:57.434530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv" event={"ID":"029948f5-1e42-453e-8016-5e11292fe3c7","Type":"ContainerStarted","Data":"58526df43214f13bf37f3f42b09f5306d677fefb337a05b6506cb50325f800b6"}
Jul 27 19:33:58.041480 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:58.041447    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:58.135998 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:58.135964    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:33:59.277439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:33:59.277397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:00.451207 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:00.451168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv" event={"ID":"029948f5-1e42-453e-8016-5e11292fe3c7","Type":"ContainerStarted","Data":"cf94bdac6d50ff823ee78e84fe61312ad9816960af3ef8209152e589a5b91764"}
Jul 27 19:34:00.470249 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:00.470202    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-766b79c75b-jt2nv" podStartSLOduration=2.05086951 podStartE2EDuration="4.470187116s" totalImagesPullingTime="2.419317606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:00.46975391 +0000 UTC m=+492.561105408" watchObservedRunningTime="2026-07-27 19:34:00.470187116 +0000 UTC m=+492.561538615"
Jul 27 19:34:02.418076 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:02.418018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:02.560668 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:02.560627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:02.885076 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:02.885046    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:02.977116 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:02.977077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:03.001891 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.001856    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"]
Jul 27 19:34:03.005175 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.005149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:03.007806 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.007784    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 27 19:34:03.007937 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.007849    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-kf8b9\""
Jul 27 19:34:03.018608 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.018577    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"]
Jul 27 19:34:03.069293 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.069261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:03.104224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.104191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcmmk\" (UniqueName: \"kubernetes.io/projected/3f7a5d90-0445-447b-b992-2628efd922f9-kube-api-access-hcmmk\") pod \"llmisvc-controller-manager-777c6bd7d5-d6vdb\" (UID: \"3f7a5d90-0445-447b-b992-2628efd922f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:03.104429 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.104276    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f7a5d90-0445-447b-b992-2628efd922f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d6vdb\" (UID: \"3f7a5d90-0445-447b-b992-2628efd922f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:03.179014 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.178981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:03.204699 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.204667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f7a5d90-0445-447b-b992-2628efd922f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d6vdb\" (UID: \"3f7a5d90-0445-447b-b992-2628efd922f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:03.204847 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.204716    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcmmk\" (UniqueName: \"kubernetes.io/projected/3f7a5d90-0445-447b-b992-2628efd922f9-kube-api-access-hcmmk\") pod \"llmisvc-controller-manager-777c6bd7d5-d6vdb\" (UID: \"3f7a5d90-0445-447b-b992-2628efd922f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:03.204847 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:34:03.204827    2579 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 27 19:34:03.204944 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:34:03.204918    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f7a5d90-0445-447b-b992-2628efd922f9-cert podName:3f7a5d90-0445-447b-b992-2628efd922f9 nodeName:}" failed. No retries permitted until 2026-07-27 19:34:03.704895444 +0000 UTC m=+495.796246937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3f7a5d90-0445-447b-b992-2628efd922f9-cert") pod "llmisvc-controller-manager-777c6bd7d5-d6vdb" (UID: "3f7a5d90-0445-447b-b992-2628efd922f9") : secret "llmisvc-webhook-server-cert" not found
Jul 27 19:34:03.219389 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.219360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcmmk\" (UniqueName: \"kubernetes.io/projected/3f7a5d90-0445-447b-b992-2628efd922f9-kube-api-access-hcmmk\") pod \"llmisvc-controller-manager-777c6bd7d5-d6vdb\" (UID: \"3f7a5d90-0445-447b-b992-2628efd922f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:03.384371 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.384340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:03.417316 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.417291    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8cbf9cd8b-t4dv2"
Jul 27 19:34:03.497748 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.497652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:03.710483 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.710451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f7a5d90-0445-447b-b992-2628efd922f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d6vdb\" (UID: \"3f7a5d90-0445-447b-b992-2628efd922f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:03.712996 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.712962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f7a5d90-0445-447b-b992-2628efd922f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d6vdb\" (UID: \"3f7a5d90-0445-447b-b992-2628efd922f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:03.915659 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:03.915566    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:04.054829 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:04.054803    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"]
Jul 27 19:34:04.057115 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:34:04.057091    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3f7a5d90_0445_447b_b992_2628efd922f9.slice/crio-5ffe5df6a3fc90e2068001eedf09081587046b53574e7ecc4677b49f5b0e9501 WatchSource:0}: Error finding container 5ffe5df6a3fc90e2068001eedf09081587046b53574e7ecc4677b49f5b0e9501: Status 404 returned error can't find the container with id 5ffe5df6a3fc90e2068001eedf09081587046b53574e7ecc4677b49f5b0e9501
Jul 27 19:34:04.208611 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:04.208578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:04.301199 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:04.301165    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:04.472136 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:04.472044    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb" event={"ID":"3f7a5d90-0445-447b-b992-2628efd922f9","Type":"ContainerStarted","Data":"5ffe5df6a3fc90e2068001eedf09081587046b53574e7ecc4677b49f5b0e9501"}
Jul 27 19:34:07.489406 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:07.489365    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb" event={"ID":"3f7a5d90-0445-447b-b992-2628efd922f9","Type":"ContainerStarted","Data":"49276c0dd229e4c5e7075cb847034f2bbc311ad3ce0f9fe187060a70126b68e9"}
Jul 27 19:34:07.489907 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:07.489486    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:07.535269 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:07.535123    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb" podStartSLOduration=2.340472638 podStartE2EDuration="5.53510516s" totalImagesPullingTime="3.194632522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:07.533076318 +0000 UTC m=+499.624427816" watchObservedRunningTime="2026-07-27 19:34:07.53510516 +0000 UTC m=+499.626456657"
Jul 27 19:34:38.495864 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:38.495838    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6vdb"
Jul 27 19:34:52.467134 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:52.467048    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:52.527448 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:52.527413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:52.662412 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:52.662361    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:53.421441 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:53.421409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:34:53.621362 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:34:53.621327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:16.083761 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:16.083721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:19.878025 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:19.877989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:22.678379 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:22.678344    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:23.032491 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:23.032394    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:24.322504 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:24.322464    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:26.512422 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:26.512390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:27.546762 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:27.546722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:28.086819 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:28.086781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:28.950512 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:28.950474    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:29.055007 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:29.054968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:31.689735 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:31.689689    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:31.817376 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:31.817343    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:35.057051 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:35.057018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:35.158333 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:35.158300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:35.330213 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:35.330124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:38.490452 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:38.490411    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:38.587067 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:38.587022    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:46.986045 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:46.986001    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:47.086919 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:47.086881    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:47.181305 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:47.181265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:48.327484 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:48.327458    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:35:48.328617 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:48.328594    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:35:48.331202 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:48.331178    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:35:48.332261 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:48.332231    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:35:49.534174 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:49.534136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:49.634420 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:49.634385    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:52.720882 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:52.720839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:52.852815 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:52.852776    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:56.332697 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:56.332653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:35:56.390615 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:35:56.390579    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:05.864932 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:05.864885    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:06.191065 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:06.191028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:06.292426 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:06.292390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:06.392510 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:06.392481    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:14.725408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:14.725322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:24.988203 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:24.988166    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:25.572283 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:25.572250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:28.219500 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:28.219464    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:40.189218 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.189182    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zht22"]
Jul 27 19:36:40.192513 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.192498    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:40.195088 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.195060    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 19:36:40.195088 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.195061    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8pc9j\""
Jul 27 19:36:40.195304 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.195099    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 19:36:40.195304 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.195099    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 27 19:36:40.198829 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.198794    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:40.200304 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.200269    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zht22"]
Jul 27 19:36:40.285441 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.285407    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zht22"]
Jul 27 19:36:40.290092 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.290064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwd6m\" (UniqueName: \"kubernetes.io/projected/061b16d7-8f5e-4f53-ad7d-89110638310b-kube-api-access-dwd6m\") pod \"limitador-limitador-64c94fd64f-zht22\" (UID: \"061b16d7-8f5e-4f53-ad7d-89110638310b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:40.290197 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.290129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/061b16d7-8f5e-4f53-ad7d-89110638310b-config-file\") pod \"limitador-limitador-64c94fd64f-zht22\" (UID: \"061b16d7-8f5e-4f53-ad7d-89110638310b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:40.391438 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.391404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwd6m\" (UniqueName: \"kubernetes.io/projected/061b16d7-8f5e-4f53-ad7d-89110638310b-kube-api-access-dwd6m\") pod \"limitador-limitador-64c94fd64f-zht22\" (UID: \"061b16d7-8f5e-4f53-ad7d-89110638310b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:40.391608 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.391445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/061b16d7-8f5e-4f53-ad7d-89110638310b-config-file\") pod \"limitador-limitador-64c94fd64f-zht22\" (UID: \"061b16d7-8f5e-4f53-ad7d-89110638310b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:40.392051 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.392033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/061b16d7-8f5e-4f53-ad7d-89110638310b-config-file\") pod \"limitador-limitador-64c94fd64f-zht22\" (UID: \"061b16d7-8f5e-4f53-ad7d-89110638310b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:40.401148 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.401119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwd6m\" (UniqueName: \"kubernetes.io/projected/061b16d7-8f5e-4f53-ad7d-89110638310b-kube-api-access-dwd6m\") pod \"limitador-limitador-64c94fd64f-zht22\" (UID: \"061b16d7-8f5e-4f53-ad7d-89110638310b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:40.504756 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.504662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:40.632555 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.632535    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zht22"]
Jul 27 19:36:40.635187 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:36:40.635160    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod061b16d7_8f5e_4f53_ad7d_89110638310b.slice/crio-0a3f8fd09a5e5aa5f44b11ff0bf6e9d807efdab54ac3d1876f568cb6d584a8e7 WatchSource:0}: Error finding container 0a3f8fd09a5e5aa5f44b11ff0bf6e9d807efdab54ac3d1876f568cb6d584a8e7: Status 404 returned error can't find the container with id 0a3f8fd09a5e5aa5f44b11ff0bf6e9d807efdab54ac3d1876f568cb6d584a8e7
Jul 27 19:36:40.987929 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.987894    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-ffqzm"]
Jul 27 19:36:40.992395 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.992372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ffqzm"
Jul 27 19:36:40.994887 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.994865    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rfc27\""
Jul 27 19:36:40.997918 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:40.997894    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ffqzm"]
Jul 27 19:36:41.096505 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:41.096466    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jb5x\" (UniqueName: \"kubernetes.io/projected/89db2f8a-88f6-4004-8768-9ba19ac5adac-kube-api-access-8jb5x\") pod \"authorino-58cd955675-ffqzm\" (UID: \"89db2f8a-88f6-4004-8768-9ba19ac5adac\") " pod="kuadrant-system/authorino-58cd955675-ffqzm"
Jul 27 19:36:41.189179 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:41.189131    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22" event={"ID":"061b16d7-8f5e-4f53-ad7d-89110638310b","Type":"ContainerStarted","Data":"0a3f8fd09a5e5aa5f44b11ff0bf6e9d807efdab54ac3d1876f568cb6d584a8e7"}
Jul 27 19:36:41.197906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:41.197840    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jb5x\" (UniqueName: \"kubernetes.io/projected/89db2f8a-88f6-4004-8768-9ba19ac5adac-kube-api-access-8jb5x\") pod \"authorino-58cd955675-ffqzm\" (UID: \"89db2f8a-88f6-4004-8768-9ba19ac5adac\") " pod="kuadrant-system/authorino-58cd955675-ffqzm"
Jul 27 19:36:41.211067 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:41.211034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jb5x\" (UniqueName: \"kubernetes.io/projected/89db2f8a-88f6-4004-8768-9ba19ac5adac-kube-api-access-8jb5x\") pod \"authorino-58cd955675-ffqzm\" (UID: \"89db2f8a-88f6-4004-8768-9ba19ac5adac\") " pod="kuadrant-system/authorino-58cd955675-ffqzm"
Jul 27 19:36:41.303852 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:41.303754    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ffqzm"
Jul 27 19:36:41.474073 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:41.473955    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ffqzm"]
Jul 27 19:36:41.476988 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:36:41.476952    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89db2f8a_88f6_4004_8768_9ba19ac5adac.slice/crio-20e12f9975518b8f9d7ceb681762d7ca3ecee19bb39d1af6dfbb0c3ea83d9c97 WatchSource:0}: Error finding container 20e12f9975518b8f9d7ceb681762d7ca3ecee19bb39d1af6dfbb0c3ea83d9c97: Status 404 returned error can't find the container with id 20e12f9975518b8f9d7ceb681762d7ca3ecee19bb39d1af6dfbb0c3ea83d9c97
Jul 27 19:36:42.107175 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:42.107126    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:42.195964 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:42.195922    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ffqzm" event={"ID":"89db2f8a-88f6-4004-8768-9ba19ac5adac","Type":"ContainerStarted","Data":"20e12f9975518b8f9d7ceb681762d7ca3ecee19bb39d1af6dfbb0c3ea83d9c97"}
Jul 27 19:36:44.206603 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:44.206572    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22" event={"ID":"061b16d7-8f5e-4f53-ad7d-89110638310b","Type":"ContainerStarted","Data":"79e52a82f6bfb4a1ea1499acc8b906f5814e06c84d838101001f07deccae218e"}
Jul 27 19:36:44.207072 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:44.206726    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:44.223387 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:44.223329    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22" podStartSLOduration=1.6280796849999999 podStartE2EDuration="4.223309758s" totalImagesPullingTime="2.595230073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:36:44.222405782 +0000 UTC m=+656.313757280" watchObservedRunningTime="2026-07-27 19:36:44.223309758 +0000 UTC m=+656.314661256"
Jul 27 19:36:45.889335 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:45.889293    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ffqzm"]
Jul 27 19:36:46.218973 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:46.218934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ffqzm" event={"ID":"89db2f8a-88f6-4004-8768-9ba19ac5adac","Type":"ContainerStarted","Data":"05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f"}
Jul 27 19:36:46.234129 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:46.234091    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-ffqzm" podStartSLOduration=2.015362232 podStartE2EDuration="6.234076108s" totalImagesPullingTime="4.218713876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:36:46.232661214 +0000 UTC m=+658.324012712" watchObservedRunningTime="2026-07-27 19:36:46.234076108 +0000 UTC m=+658.325427606"
Jul 27 19:36:47.228266 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:47.225967    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-ffqzm" podUID="89db2f8a-88f6-4004-8768-9ba19ac5adac" containerName="authorino" containerID="cri-o://05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f" gracePeriod=30
Jul 27 19:36:47.482179 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:47.482118    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ffqzm"
Jul 27 19:36:47.558000 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:47.557967    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jb5x\" (UniqueName: \"kubernetes.io/projected/89db2f8a-88f6-4004-8768-9ba19ac5adac-kube-api-access-8jb5x\") pod \"89db2f8a-88f6-4004-8768-9ba19ac5adac\" (UID: \"89db2f8a-88f6-4004-8768-9ba19ac5adac\") "
Jul 27 19:36:47.560311 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:47.560285    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89db2f8a-88f6-4004-8768-9ba19ac5adac-kube-api-access-8jb5x" (OuterVolumeSpecName: "kube-api-access-8jb5x") pod "89db2f8a-88f6-4004-8768-9ba19ac5adac" (UID: "89db2f8a-88f6-4004-8768-9ba19ac5adac"). InnerVolumeSpecName "kube-api-access-8jb5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:36:47.659436 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:47.659394    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jb5x\" (UniqueName: \"kubernetes.io/projected/89db2f8a-88f6-4004-8768-9ba19ac5adac-kube-api-access-8jb5x\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:36:48.232344 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.232313    2579 generic.go:356] "Generic (PLEG): container finished" podID="89db2f8a-88f6-4004-8768-9ba19ac5adac" containerID="05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f" exitCode=0
Jul 27 19:36:48.232778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.232351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ffqzm" event={"ID":"89db2f8a-88f6-4004-8768-9ba19ac5adac","Type":"ContainerDied","Data":"05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f"}
Jul 27 19:36:48.232778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.232367    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ffqzm"
Jul 27 19:36:48.232778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.232378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ffqzm" event={"ID":"89db2f8a-88f6-4004-8768-9ba19ac5adac","Type":"ContainerDied","Data":"20e12f9975518b8f9d7ceb681762d7ca3ecee19bb39d1af6dfbb0c3ea83d9c97"}
Jul 27 19:36:48.232778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.232395    2579 scope.go:117] "RemoveContainer" containerID="05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f"
Jul 27 19:36:48.241864 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.241846    2579 scope.go:117] "RemoveContainer" containerID="05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f"
Jul 27 19:36:48.242109 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:36:48.242088    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f\": container with ID starting with 05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f not found: ID does not exist" containerID="05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f"
Jul 27 19:36:48.242177 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.242117    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f"} err="failed to get container status \"05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f\": rpc error: code = NotFound desc = could not find container \"05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f\": container with ID starting with 05ed788587d8262a6f10936b0d3d58713f344342f4729b267b50cdd705d5283f not found: ID does not exist"
Jul 27 19:36:48.253467 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.253444    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ffqzm"]
Jul 27 19:36:48.258605 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.258584    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-ffqzm"]
Jul 27 19:36:48.490287 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:48.490181    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89db2f8a-88f6-4004-8768-9ba19ac5adac" path="/var/lib/kubelet/pods/89db2f8a-88f6-4004-8768-9ba19ac5adac/volumes"
Jul 27 19:36:55.214735 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:55.214705    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:56.890567 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:56.890531    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zht22"]
Jul 27 19:36:56.891085 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:56.890721    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22" podUID="061b16d7-8f5e-4f53-ad7d-89110638310b" containerName="limitador" containerID="cri-o://79e52a82f6bfb4a1ea1499acc8b906f5814e06c84d838101001f07deccae218e" gracePeriod=30
Jul 27 19:36:57.273149 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:57.273117    2579 generic.go:356] "Generic (PLEG): container finished" podID="061b16d7-8f5e-4f53-ad7d-89110638310b" containerID="79e52a82f6bfb4a1ea1499acc8b906f5814e06c84d838101001f07deccae218e" exitCode=0
Jul 27 19:36:57.273344 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:57.273177    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22" event={"ID":"061b16d7-8f5e-4f53-ad7d-89110638310b","Type":"ContainerDied","Data":"79e52a82f6bfb4a1ea1499acc8b906f5814e06c84d838101001f07deccae218e"}
Jul 27 19:36:57.843943 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:57.843919    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:57.957834 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:57.957748    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/061b16d7-8f5e-4f53-ad7d-89110638310b-config-file\") pod \"061b16d7-8f5e-4f53-ad7d-89110638310b\" (UID: \"061b16d7-8f5e-4f53-ad7d-89110638310b\") "
Jul 27 19:36:57.957834 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:57.957834    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwd6m\" (UniqueName: \"kubernetes.io/projected/061b16d7-8f5e-4f53-ad7d-89110638310b-kube-api-access-dwd6m\") pod \"061b16d7-8f5e-4f53-ad7d-89110638310b\" (UID: \"061b16d7-8f5e-4f53-ad7d-89110638310b\") "
Jul 27 19:36:57.958215 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:57.958146    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/061b16d7-8f5e-4f53-ad7d-89110638310b-config-file" (OuterVolumeSpecName: "config-file") pod "061b16d7-8f5e-4f53-ad7d-89110638310b" (UID: "061b16d7-8f5e-4f53-ad7d-89110638310b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:36:57.959712 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:57.959689    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/061b16d7-8f5e-4f53-ad7d-89110638310b-kube-api-access-dwd6m" (OuterVolumeSpecName: "kube-api-access-dwd6m") pod "061b16d7-8f5e-4f53-ad7d-89110638310b" (UID: "061b16d7-8f5e-4f53-ad7d-89110638310b"). InnerVolumeSpecName "kube-api-access-dwd6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:36:58.059236 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.059203    2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/061b16d7-8f5e-4f53-ad7d-89110638310b-config-file\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:36:58.059236 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.059230    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwd6m\" (UniqueName: \"kubernetes.io/projected/061b16d7-8f5e-4f53-ad7d-89110638310b-kube-api-access-dwd6m\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:36:58.269281 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.269181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:58.279068 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.279044    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22"
Jul 27 19:36:58.279219 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.279075    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zht22" event={"ID":"061b16d7-8f5e-4f53-ad7d-89110638310b","Type":"ContainerDied","Data":"0a3f8fd09a5e5aa5f44b11ff0bf6e9d807efdab54ac3d1876f568cb6d584a8e7"}
Jul 27 19:36:58.279219 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.279136    2579 scope.go:117] "RemoveContainer" containerID="79e52a82f6bfb4a1ea1499acc8b906f5814e06c84d838101001f07deccae218e"
Jul 27 19:36:58.300461 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.300437    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zht22"]
Jul 27 19:36:58.304763 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.304739    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zht22"]
Jul 27 19:36:58.367410 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.367355    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:58.488327 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.488295    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="061b16d7-8f5e-4f53-ad7d-89110638310b" path="/var/lib/kubelet/pods/061b16d7-8f5e-4f53-ad7d-89110638310b/volumes"
Jul 27 19:36:58.765408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.765369    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"]
Jul 27 19:36:58.765942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.765924    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89db2f8a-88f6-4004-8768-9ba19ac5adac" containerName="authorino"
Jul 27 19:36:58.766014 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.765944    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="89db2f8a-88f6-4004-8768-9ba19ac5adac" containerName="authorino"
Jul 27 19:36:58.766014 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.765967    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="061b16d7-8f5e-4f53-ad7d-89110638310b" containerName="limitador"
Jul 27 19:36:58.766014 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.765976    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="061b16d7-8f5e-4f53-ad7d-89110638310b" containerName="limitador"
Jul 27 19:36:58.766115 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.766055    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="89db2f8a-88f6-4004-8768-9ba19ac5adac" containerName="authorino"
Jul 27 19:36:58.766115 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.766068    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="061b16d7-8f5e-4f53-ad7d-89110638310b" containerName="limitador"
Jul 27 19:36:58.769404 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.769381    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:36:58.772337 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.772301    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-brj2t\""
Jul 27 19:36:58.772337 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.772310    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 27 19:36:58.773173 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.773153    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:36:58.773285 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.773196    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 27 19:36:58.776498 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.776476    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"]
Jul 27 19:36:58.802900 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.802872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:36:58.866635 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.866602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff-data\") pod \"postgres-69fb4594cf-dvbks\" (UID: \"46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:36:58.866808 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.866703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq9mh\" (UniqueName: \"kubernetes.io/projected/46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff-kube-api-access-mq9mh\") pod \"postgres-69fb4594cf-dvbks\" (UID: \"46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:36:58.967260 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.967200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff-data\") pod \"postgres-69fb4594cf-dvbks\" (UID: \"46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:36:58.967693 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.967388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq9mh\" (UniqueName: \"kubernetes.io/projected/46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff-kube-api-access-mq9mh\") pod \"postgres-69fb4594cf-dvbks\" (UID: \"46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:36:58.967693 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.967602    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff-data\") pod \"postgres-69fb4594cf-dvbks\" (UID: \"46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:36:58.975481 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:58.975451    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq9mh\" (UniqueName: \"kubernetes.io/projected/46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff-kube-api-access-mq9mh\") pod \"postgres-69fb4594cf-dvbks\" (UID: \"46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:36:59.080145 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:59.080047    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:36:59.208264 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:59.208225    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"]
Jul 27 19:36:59.210423 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:36:59.210394    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46db5e5c_3a3c_44c5_b2c9_bd7b484b2eff.slice/crio-8fdc0853b4cd388fe75e8ed0d172fde0f8574437448fd0e0625d1494ffad8962 WatchSource:0}: Error finding container 8fdc0853b4cd388fe75e8ed0d172fde0f8574437448fd0e0625d1494ffad8962: Status 404 returned error can't find the container with id 8fdc0853b4cd388fe75e8ed0d172fde0f8574437448fd0e0625d1494ffad8962
Jul 27 19:36:59.286435 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:59.286401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks" event={"ID":"46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff","Type":"ContainerStarted","Data":"8fdc0853b4cd388fe75e8ed0d172fde0f8574437448fd0e0625d1494ffad8962"}
Jul 27 19:36:59.663373 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:36:59.663334    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:04.318222 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:04.318137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks" event={"ID":"46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff","Type":"ContainerStarted","Data":"13770ac8a66137844ac2124bcc7abd752b90b3e33e91e705677265d279c213b4"}
Jul 27 19:37:04.318614 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:04.318221    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:37:04.337142 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:04.337095    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks" podStartSLOduration=1.529951879 podStartE2EDuration="6.337081278s" totalImagesPullingTime="4.807129399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:04.336063834 +0000 UTC m=+676.427415336" watchObservedRunningTime="2026-07-27 19:37:04.337081278 +0000 UTC m=+676.428432776"
Jul 27 19:37:10.350764 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:10.350737    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dvbks"
Jul 27 19:37:10.688710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:10.688676    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:10.914882 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:10.914851    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-d72l5"]
Jul 27 19:37:10.920193 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:10.920170    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-d72l5"
Jul 27 19:37:10.922813 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:10.922790    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 19:37:10.923950 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:10.923930    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 19:37:10.923950 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:10.923946    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rfc27\""
Jul 27 19:37:10.926428 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:10.926397    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-d72l5"]
Jul 27 19:37:11.078144 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.078044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb85f\" (UniqueName: \"kubernetes.io/projected/db92d88f-a816-4002-832b-312bb4a73710-kube-api-access-qb85f\") pod \"authorino-5d4b67c446-d72l5\" (UID: \"db92d88f-a816-4002-832b-312bb4a73710\") " pod="kuadrant-system/authorino-5d4b67c446-d72l5"
Jul 27 19:37:11.143907 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.143873    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-d72l5"]
Jul 27 19:37:11.144158 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:37:11.144139    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qb85f], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-d72l5" podUID="db92d88f-a816-4002-832b-312bb4a73710"
Jul 27 19:37:11.169824 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.169789    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-585678cbf5-2tkwh"]
Jul 27 19:37:11.173148 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.173131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-585678cbf5-2tkwh"
Jul 27 19:37:11.178843 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.178811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb85f\" (UniqueName: \"kubernetes.io/projected/db92d88f-a816-4002-832b-312bb4a73710-kube-api-access-qb85f\") pod \"authorino-5d4b67c446-d72l5\" (UID: \"db92d88f-a816-4002-832b-312bb4a73710\") " pod="kuadrant-system/authorino-5d4b67c446-d72l5"
Jul 27 19:37:11.180815 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.180797    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-585678cbf5-2tkwh"]
Jul 27 19:37:11.186666 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.186644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb85f\" (UniqueName: \"kubernetes.io/projected/db92d88f-a816-4002-832b-312bb4a73710-kube-api-access-qb85f\") pod \"authorino-5d4b67c446-d72l5\" (UID: \"db92d88f-a816-4002-832b-312bb4a73710\") " pod="kuadrant-system/authorino-5d4b67c446-d72l5"
Jul 27 19:37:11.200318 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.200292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:11.279422 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.279385    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcpwl\" (UniqueName: \"kubernetes.io/projected/e93c5b12-c354-4239-bd65-342ceb973494-kube-api-access-xcpwl\") pod \"authorino-585678cbf5-2tkwh\" (UID: \"e93c5b12-c354-4239-bd65-342ceb973494\") " pod="kuadrant-system/authorino-585678cbf5-2tkwh"
Jul 27 19:37:11.352163 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.352089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-d72l5"
Jul 27 19:37:11.361050 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.361022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-d72l5"
Jul 27 19:37:11.379829 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.379799    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-585678cbf5-2tkwh"]
Jul 27 19:37:11.380062 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:37:11.380043    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xcpwl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-585678cbf5-2tkwh" podUID="e93c5b12-c354-4239-bd65-342ceb973494"
Jul 27 19:37:11.380136 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.380091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcpwl\" (UniqueName: \"kubernetes.io/projected/e93c5b12-c354-4239-bd65-342ceb973494-kube-api-access-xcpwl\") pod \"authorino-585678cbf5-2tkwh\" (UID: \"e93c5b12-c354-4239-bd65-342ceb973494\") " pod="kuadrant-system/authorino-585678cbf5-2tkwh"
Jul 27 19:37:11.389664 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.389641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcpwl\" (UniqueName: \"kubernetes.io/projected/e93c5b12-c354-4239-bd65-342ceb973494-kube-api-access-xcpwl\") pod \"authorino-585678cbf5-2tkwh\" (UID: \"e93c5b12-c354-4239-bd65-342ceb973494\") " pod="kuadrant-system/authorino-585678cbf5-2tkwh"
Jul 27 19:37:11.406201 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.406172    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5ddbb5c7cf-22c4x"]
Jul 27 19:37:11.410360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.410340    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:37:11.412870 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.412847    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 27 19:37:11.419100 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.419075    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ddbb5c7cf-22c4x"]
Jul 27 19:37:11.480817 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.480786    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qb85f\" (UniqueName: \"kubernetes.io/projected/db92d88f-a816-4002-832b-312bb4a73710-kube-api-access-qb85f\") pod \"db92d88f-a816-4002-832b-312bb4a73710\" (UID: \"db92d88f-a816-4002-832b-312bb4a73710\") "
Jul 27 19:37:11.483073 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.483048    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db92d88f-a816-4002-832b-312bb4a73710-kube-api-access-qb85f" (OuterVolumeSpecName: "kube-api-access-qb85f") pod "db92d88f-a816-4002-832b-312bb4a73710" (UID: "db92d88f-a816-4002-832b-312bb4a73710"). InnerVolumeSpecName "kube-api-access-qb85f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:37:11.582210 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.582174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a02b0e3-dc52-4001-a78c-603754bc4453-tls-cert\") pod \"authorino-5ddbb5c7cf-22c4x\" (UID: \"3a02b0e3-dc52-4001-a78c-603754bc4453\") " pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:37:11.582409 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.582279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnkdm\" (UniqueName: \"kubernetes.io/projected/3a02b0e3-dc52-4001-a78c-603754bc4453-kube-api-access-dnkdm\") pod \"authorino-5ddbb5c7cf-22c4x\" (UID: \"3a02b0e3-dc52-4001-a78c-603754bc4453\") " pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:37:11.582409 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.582314    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb85f\" (UniqueName: \"kubernetes.io/projected/db92d88f-a816-4002-832b-312bb4a73710-kube-api-access-qb85f\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:37:11.683199 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.683170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnkdm\" (UniqueName: \"kubernetes.io/projected/3a02b0e3-dc52-4001-a78c-603754bc4453-kube-api-access-dnkdm\") pod \"authorino-5ddbb5c7cf-22c4x\" (UID: \"3a02b0e3-dc52-4001-a78c-603754bc4453\") " pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:37:11.683366 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.683225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a02b0e3-dc52-4001-a78c-603754bc4453-tls-cert\") pod \"authorino-5ddbb5c7cf-22c4x\" (UID: \"3a02b0e3-dc52-4001-a78c-603754bc4453\") " pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:37:11.685683 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.685651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a02b0e3-dc52-4001-a78c-603754bc4453-tls-cert\") pod \"authorino-5ddbb5c7cf-22c4x\" (UID: \"3a02b0e3-dc52-4001-a78c-603754bc4453\") " pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:37:11.691225 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.691201    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnkdm\" (UniqueName: \"kubernetes.io/projected/3a02b0e3-dc52-4001-a78c-603754bc4453-kube-api-access-dnkdm\") pod \"authorino-5ddbb5c7cf-22c4x\" (UID: \"3a02b0e3-dc52-4001-a78c-603754bc4453\") " pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:37:11.720481 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.720454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:37:11.855225 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:11.855198    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ddbb5c7cf-22c4x"]
Jul 27 19:37:11.857533 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:37:11.857506    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a02b0e3_dc52_4001_a78c_603754bc4453.slice/crio-2b7e2beee7adea01fb1035534f514516411a8917e8a7e59c5ac5650f1ff252bf WatchSource:0}: Error finding container 2b7e2beee7adea01fb1035534f514516411a8917e8a7e59c5ac5650f1ff252bf: Status 404 returned error can't find the container with id 2b7e2beee7adea01fb1035534f514516411a8917e8a7e59c5ac5650f1ff252bf
Jul 27 19:37:12.357498 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.357458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x" event={"ID":"3a02b0e3-dc52-4001-a78c-603754bc4453","Type":"ContainerStarted","Data":"2b7e2beee7adea01fb1035534f514516411a8917e8a7e59c5ac5650f1ff252bf"}
Jul 27 19:37:12.357498 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.357485    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-585678cbf5-2tkwh"
Jul 27 19:37:12.357938 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.357489    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-d72l5"
Jul 27 19:37:12.398929 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.398903    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-585678cbf5-2tkwh"
Jul 27 19:37:12.421695 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.421663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-d72l5"]
Jul 27 19:37:12.423153 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.423132    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-d72l5"]
Jul 27 19:37:12.488790 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.488756    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db92d88f-a816-4002-832b-312bb4a73710" path="/var/lib/kubelet/pods/db92d88f-a816-4002-832b-312bb4a73710/volumes"
Jul 27 19:37:12.591476 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.591458    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcpwl\" (UniqueName: \"kubernetes.io/projected/e93c5b12-c354-4239-bd65-342ceb973494-kube-api-access-xcpwl\") pod \"e93c5b12-c354-4239-bd65-342ceb973494\" (UID: \"e93c5b12-c354-4239-bd65-342ceb973494\") "
Jul 27 19:37:12.593379 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.593351    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93c5b12-c354-4239-bd65-342ceb973494-kube-api-access-xcpwl" (OuterVolumeSpecName: "kube-api-access-xcpwl") pod "e93c5b12-c354-4239-bd65-342ceb973494" (UID: "e93c5b12-c354-4239-bd65-342ceb973494"). InnerVolumeSpecName "kube-api-access-xcpwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:37:12.692001 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:12.691977    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcpwl\" (UniqueName: \"kubernetes.io/projected/e93c5b12-c354-4239-bd65-342ceb973494-kube-api-access-xcpwl\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:37:13.363255 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:13.363223    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-585678cbf5-2tkwh"
Jul 27 19:37:13.363638 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:13.363217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x" event={"ID":"3a02b0e3-dc52-4001-a78c-603754bc4453","Type":"ContainerStarted","Data":"2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18"}
Jul 27 19:37:13.382868 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:13.382820    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x" podStartSLOduration=1.659224943 podStartE2EDuration="2.382805259s" totalImagesPullingTime="723.580316ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:13.380842963 +0000 UTC m=+685.472194460" watchObservedRunningTime="2026-07-27 19:37:13.382805259 +0000 UTC m=+685.474156757"
Jul 27 19:37:13.404672 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:13.404643    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-585678cbf5-2tkwh"]
Jul 27 19:37:13.410991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:13.410964    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-585678cbf5-2tkwh"]
Jul 27 19:37:14.489056 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:14.489027    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e93c5b12-c354-4239-bd65-342ceb973494" path="/var/lib/kubelet/pods/e93c5b12-c354-4239-bd65-342ceb973494/volumes"
Jul 27 19:37:18.005303 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.005269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:18.140198 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.140164    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:18.218023 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.217989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:18.221709 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.221685    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5744b65d6-fwxnr"]
Jul 27 19:37:18.238561 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.238536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5744b65d6-fwxnr"]
Jul 27 19:37:18.238728 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.238626    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:18.241653 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.241630    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jcp46\""
Jul 27 19:37:18.241957 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.241930    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 19:37:18.242415 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.242396    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 19:37:18.245760 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.245739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dnjk\" (UniqueName: \"kubernetes.io/projected/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-kube-api-access-7dnjk\") pod \"maas-controller-5744b65d6-fwxnr\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") " pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:18.245876 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.245775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert\") pod \"maas-controller-5744b65d6-fwxnr\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") " pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:18.346268 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.346165    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dnjk\" (UniqueName: \"kubernetes.io/projected/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-kube-api-access-7dnjk\") pod \"maas-controller-5744b65d6-fwxnr\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") " pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:18.346268 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.346195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert\") pod \"maas-controller-5744b65d6-fwxnr\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") " pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:18.346460 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:37:18.346351    2579 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 27 19:37:18.346460 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:37:18.346408    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert podName:e424a6b4-96b1-4bdc-8fcb-2d0c2a224267 nodeName:}" failed. No retries permitted until 2026-07-27 19:37:18.846390986 +0000 UTC m=+690.937742461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert") pod "maas-controller-5744b65d6-fwxnr" (UID: "e424a6b4-96b1-4bdc-8fcb-2d0c2a224267") : secret "maas-controller-webhook-cert" not found
Jul 27 19:37:18.355014 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.354981    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dnjk\" (UniqueName: \"kubernetes.io/projected/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-kube-api-access-7dnjk\") pod \"maas-controller-5744b65d6-fwxnr\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") " pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:18.412039 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.412003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:18.849467 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.849440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert\") pod \"maas-controller-5744b65d6-fwxnr\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") " pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:18.851777 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:18.851744    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert\") pod \"maas-controller-5744b65d6-fwxnr\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") " pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:19.149082 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:19.148999    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:19.275611 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:19.275583    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5744b65d6-fwxnr"]
Jul 27 19:37:19.277945 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:37:19.277917    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode424a6b4_96b1_4bdc_8fcb_2d0c2a224267.slice/crio-cd0222a8b332202dacd0834e42a611fbc0ec5ccb8b458623a29d0d0a780e3544 WatchSource:0}: Error finding container cd0222a8b332202dacd0834e42a611fbc0ec5ccb8b458623a29d0d0a780e3544: Status 404 returned error can't find the container with id cd0222a8b332202dacd0834e42a611fbc0ec5ccb8b458623a29d0d0a780e3544
Jul 27 19:37:19.394381 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:19.394347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-fwxnr" event={"ID":"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267","Type":"ContainerStarted","Data":"cd0222a8b332202dacd0834e42a611fbc0ec5ccb8b458623a29d0d0a780e3544"}
Jul 27 19:37:21.396575 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:21.396536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:21.497314 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:21.497288    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:22.410765 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:22.410732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-fwxnr" event={"ID":"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267","Type":"ContainerStarted","Data":"01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc"}
Jul 27 19:37:22.411178 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:22.410854    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:22.428033 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:22.427979    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5744b65d6-fwxnr" podStartSLOduration=2.219981944 podStartE2EDuration="4.427958271s" totalImagesPullingTime="2.207976327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:22.427766822 +0000 UTC m=+694.519118400" watchObservedRunningTime="2026-07-27 19:37:22.427958271 +0000 UTC m=+694.519309772"
Jul 27 19:37:25.735129 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:25.735095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:33.433744 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:33.433708    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:37:53.330393 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:53.330320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:53.429280 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:53.429223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:54.025089 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.025053    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:54.157874 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.157836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:54.478297 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.478265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:54.545881 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.545848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:37:54.573004 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.572969    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"]
Jul 27 19:37:54.577854 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.577830    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:54.580504 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.580485    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 27 19:37:54.587068 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.587049    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"]
Jul 27 19:37:54.658488 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.658452    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26d2d9cd-b794-4ae7-97e9-d5bb72da28af-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-82txl\" (UID: \"26d2d9cd-b794-4ae7-97e9-d5bb72da28af\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:54.658661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.658586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxqln\" (UniqueName: \"kubernetes.io/projected/26d2d9cd-b794-4ae7-97e9-d5bb72da28af-kube-api-access-zxqln\") pod \"payload-pre-processing-687dbb6cf6-82txl\" (UID: \"26d2d9cd-b794-4ae7-97e9-d5bb72da28af\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:54.759922 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.759833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxqln\" (UniqueName: \"kubernetes.io/projected/26d2d9cd-b794-4ae7-97e9-d5bb72da28af-kube-api-access-zxqln\") pod \"payload-pre-processing-687dbb6cf6-82txl\" (UID: \"26d2d9cd-b794-4ae7-97e9-d5bb72da28af\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:54.759922 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.759882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26d2d9cd-b794-4ae7-97e9-d5bb72da28af-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-82txl\" (UID: \"26d2d9cd-b794-4ae7-97e9-d5bb72da28af\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:54.760552 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.760533    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26d2d9cd-b794-4ae7-97e9-d5bb72da28af-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-82txl\" (UID: \"26d2d9cd-b794-4ae7-97e9-d5bb72da28af\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:54.767940 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.767901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxqln\" (UniqueName: \"kubernetes.io/projected/26d2d9cd-b794-4ae7-97e9-d5bb72da28af-kube-api-access-zxqln\") pod \"payload-pre-processing-687dbb6cf6-82txl\" (UID: \"26d2d9cd-b794-4ae7-97e9-d5bb72da28af\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:54.889314 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:54.889276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:55.043465 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:55.043420    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"]
Jul 27 19:37:55.046126 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:37:55.046096    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d2d9cd_b794_4ae7_97e9_d5bb72da28af.slice/crio-e0712eef335dac73dab6e2fbc332f54c68e72edaf24db8dda8cfc69509f3e45e WatchSource:0}: Error finding container e0712eef335dac73dab6e2fbc332f54c68e72edaf24db8dda8cfc69509f3e45e: Status 404 returned error can't find the container with id e0712eef335dac73dab6e2fbc332f54c68e72edaf24db8dda8cfc69509f3e45e
Jul 27 19:37:55.564901 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:55.564864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl" event={"ID":"26d2d9cd-b794-4ae7-97e9-d5bb72da28af","Type":"ContainerStarted","Data":"e0712eef335dac73dab6e2fbc332f54c68e72edaf24db8dda8cfc69509f3e45e"}
Jul 27 19:37:56.571331 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:56.571292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl" event={"ID":"26d2d9cd-b794-4ae7-97e9-d5bb72da28af","Type":"ContainerStarted","Data":"b1fc70fa80c46a37d5c745f259d18c48072893de0763daa53091ec98b04a36ec"}
Jul 27 19:37:56.571745 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:56.571390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:37:56.588831 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:37:56.588779    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl" podStartSLOduration=1.235707357 podStartE2EDuration="2.588761881s" totalImagesPullingTime="1.353054524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:56.587011648 +0000 UTC m=+728.678363146" watchObservedRunningTime="2026-07-27 19:37:56.588761881 +0000 UTC m=+728.680113413"
Jul 27 19:38:04.455442 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:04.455400    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:04.554515 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:04.554476    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:07.578400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:07.578373    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82txl"
Jul 27 19:38:12.800735 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.800707    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"]
Jul 27 19:38:12.803726 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.803693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:12.807447 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.807424    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 19:38:12.807447 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.807436    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 19:38:12.807655 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.807426    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pt647\""
Jul 27 19:38:12.807655 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.807427    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 27 19:38:12.814420 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.814398    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"]
Jul 27 19:38:12.823970 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.823948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:12.928821 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.928790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:12.928993 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.928908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cr7x\" (UniqueName: \"kubernetes.io/projected/3bd7c5c8-286b-4ef8-b54e-99a212687321-kube-api-access-7cr7x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:12.928993 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.928951    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bd7c5c8-286b-4ef8-b54e-99a212687321-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:12.929087 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.928995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:12.929123 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.929087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:12.929165 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:12.929130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.029794 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.029758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.029982 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.029859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cr7x\" (UniqueName: \"kubernetes.io/projected/3bd7c5c8-286b-4ef8-b54e-99a212687321-kube-api-access-7cr7x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.029982 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.029889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bd7c5c8-286b-4ef8-b54e-99a212687321-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.029982 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.029928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.029982 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.029959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.030186 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.029986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.030400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.030373    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.030400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.030391    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.030561 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.030473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.032215 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.032192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bd7c5c8-286b-4ef8-b54e-99a212687321-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.032315 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.032300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bd7c5c8-286b-4ef8-b54e-99a212687321-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.037782 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.037757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cr7x\" (UniqueName: \"kubernetes.io/projected/3bd7c5c8-286b-4ef8-b54e-99a212687321-kube-api-access-7cr7x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk\" (UID: \"3bd7c5c8-286b-4ef8-b54e-99a212687321\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.117584 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.117488    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:13.256190 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.256165    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"]
Jul 27 19:38:13.259214 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:38:13.259188    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd7c5c8_286b_4ef8_b54e_99a212687321.slice/crio-134d03a1610c9ea41ef3bd26d9e0898e414a5890895d3a274cc93db0cf0acf55 WatchSource:0}: Error finding container 134d03a1610c9ea41ef3bd26d9e0898e414a5890895d3a274cc93db0cf0acf55: Status 404 returned error can't find the container with id 134d03a1610c9ea41ef3bd26d9e0898e414a5890895d3a274cc93db0cf0acf55
Jul 27 19:38:13.260872 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.260849    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:38:13.655780 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:13.655738    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk" event={"ID":"3bd7c5c8-286b-4ef8-b54e-99a212687321","Type":"ContainerStarted","Data":"134d03a1610c9ea41ef3bd26d9e0898e414a5890895d3a274cc93db0cf0acf55"}
Jul 27 19:38:14.661765 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:14.661672    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk" event={"ID":"3bd7c5c8-286b-4ef8-b54e-99a212687321","Type":"ContainerStarted","Data":"767fc7b3fe2f1af23c64f38447799efdddaa6f96cdee2e59fb339e106794efe7"}
Jul 27 19:38:14.681824 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:14.681778    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk" podStartSLOduration=1.564657945 podStartE2EDuration="2.681763302s" totalImagesPullingTime="1.117105357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:38:14.679345673 +0000 UTC m=+746.770697171" watchObservedRunningTime="2026-07-27 19:38:14.681763302 +0000 UTC m=+746.773114799"
Jul 27 19:38:23.117862 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:23.117825    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:23.118303 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:23.117895    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:23.130010 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:23.129989    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:23.713389 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:23.713358    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk"
Jul 27 19:38:24.818889 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:24.818848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:34.708619 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:34.708571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:34.809360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:34.809307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:34.892678 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:34.892648    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"]
Jul 27 19:38:34.896136 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:34.896118    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:34.898685 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:34.898661    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 27 19:38:34.904651 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:34.904631    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"]
Jul 27 19:38:34.924850 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:34.924822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:35.029106 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.029012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.029106 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.029078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.029106 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.029104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq26r\" (UniqueName: \"kubernetes.io/projected/05394dcf-a888-42bd-9ba7-04440a5e6c09-kube-api-access-cq26r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.029422 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.029188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.029422 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.029226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05394dcf-a888-42bd-9ba7-04440a5e6c09-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.029422 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.029316    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.129942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.129885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.130150 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.130055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq26r\" (UniqueName: \"kubernetes.io/projected/05394dcf-a888-42bd-9ba7-04440a5e6c09-kube-api-access-cq26r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.130150 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.130131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.130321 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.130163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05394dcf-a888-42bd-9ba7-04440a5e6c09-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.130321 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.130221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.130321 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.130300    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.130554 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.130529    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.130683 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.130661    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.130740 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.130704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.132219 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.132197    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05394dcf-a888-42bd-9ba7-04440a5e6c09-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.132712 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.132694    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05394dcf-a888-42bd-9ba7-04440a5e6c09-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.137084 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.137060    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq26r\" (UniqueName: \"kubernetes.io/projected/05394dcf-a888-42bd-9ba7-04440a5e6c09-kube-api-access-cq26r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-spj8b\" (UID: \"05394dcf-a888-42bd-9ba7-04440a5e6c09\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.207906 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.207873    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:35.338949 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.338923    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"]
Jul 27 19:38:35.341370 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:38:35.341335    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05394dcf_a888_42bd_9ba7_04440a5e6c09.slice/crio-0f7d06c52519ec949fa09aff066425c11f280bd515b6d1ac4dee7cac5da767f5 WatchSource:0}: Error finding container 0f7d06c52519ec949fa09aff066425c11f280bd515b6d1ac4dee7cac5da767f5: Status 404 returned error can't find the container with id 0f7d06c52519ec949fa09aff066425c11f280bd515b6d1ac4dee7cac5da767f5
Jul 27 19:38:35.761999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.761961    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b" event={"ID":"05394dcf-a888-42bd-9ba7-04440a5e6c09","Type":"ContainerStarted","Data":"e631bb19f50a7923d3926283a907f8c4654500a0b6a028b3e4c4af340f5d5c74"}
Jul 27 19:38:35.762437 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.762023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b" event={"ID":"05394dcf-a888-42bd-9ba7-04440a5e6c09","Type":"ContainerStarted","Data":"0f7d06c52519ec949fa09aff066425c11f280bd515b6d1ac4dee7cac5da767f5"}
Jul 27 19:38:35.779066 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:35.779018    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b" podStartSLOduration=1.616759691 podStartE2EDuration="1.779002312s" totalImagesPullingTime="162.242621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:38:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:38:35.7779987 +0000 UTC m=+767.869350199" watchObservedRunningTime="2026-07-27 19:38:35.779002312 +0000 UTC m=+767.870353809"
Jul 27 19:38:45.208219 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:45.208178    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:45.208758 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:45.208274    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:45.220641 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:45.220613    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:45.822714 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:45.822677    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-spj8b"
Jul 27 19:38:48.830199 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:48.830164    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:59.275869 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:59.275825    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:38:59.345314 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:38:59.345280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:06.052524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:06.052487    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:22.091999 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.091918    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"]
Jul 27 19:39:22.095212 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.095191    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.097978 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.097957    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 27 19:39:22.104322 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.104295    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"]
Jul 27 19:39:22.129688 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.129645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:39:22.148767 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.148735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.148905 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.148804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.148905 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.148830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rbrl\" (UniqueName: \"kubernetes.io/projected/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-kube-api-access-5rbrl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.148905 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.148855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.148905 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.148874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.149047 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.148964    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250212 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250212 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250484 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250484 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250313    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250484 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250484 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rbrl\" (UniqueName: \"kubernetes.io/projected/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-kube-api-access-5rbrl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250816 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250793    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250903 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250812    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.250903 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.250865    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.252490 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.252467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.252735 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.252717    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.258103 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.258081    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rbrl\" (UniqueName: \"kubernetes.io/projected/07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800-kube-api-access-5rbrl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5\" (UID: \"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.406717 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.406621    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:22.538561 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.538536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"]
Jul 27 19:39:22.540589 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:39:22.540556    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07b7d2d7_d7ba_44c9_95c5_03b8e5ed9800.slice/crio-e324406278c8ba941655e33c5d128bf8ef01a53ac254ee22465ba7811ff574f1 WatchSource:0}: Error finding container e324406278c8ba941655e33c5d128bf8ef01a53ac254ee22465ba7811ff574f1: Status 404 returned error can't find the container with id e324406278c8ba941655e33c5d128bf8ef01a53ac254ee22465ba7811ff574f1
Jul 27 19:39:22.994772 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.994738    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5" event={"ID":"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800","Type":"ContainerStarted","Data":"31e94ab7cd7afa6bbbed9fb5c5b095e684122f78511c50c71d252421dfaf54ad"}
Jul 27 19:39:22.994937 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:22.994782    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5" event={"ID":"07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800","Type":"ContainerStarted","Data":"e324406278c8ba941655e33c5d128bf8ef01a53ac254ee22465ba7811ff574f1"}
Jul 27 19:39:23.011339 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:23.011286    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5" podStartSLOduration=0.696541315 podStartE2EDuration="1.011268804s" totalImagesPullingTime="314.727489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:23.010347521 +0000 UTC m=+815.101699019" watchObservedRunningTime="2026-07-27 19:39:23.011268804 +0000 UTC m=+815.102620299"
Jul 27 19:39:32.407752 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:32.407705    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:32.408159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:32.407771    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:32.420671 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:32.420642    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:39:33.056019 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:39:33.055994    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5"
Jul 27 19:40:40.793093 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:40.793008    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:40.895602 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:40.895565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:41.948019 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:41.947981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:42.031032 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.031000    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ddbb5c7cf-22c4x"]
Jul 27 19:40:42.031219 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.031196    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x" podUID="3a02b0e3-dc52-4001-a78c-603754bc4453" containerName="authorino" containerID="cri-o://2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18" gracePeriod=30
Jul 27 19:40:42.042321 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.042285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:42.283202 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.283180    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:40:42.283694 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.283675    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a02b0e3-dc52-4001-a78c-603754bc4453-tls-cert\") pod \"3a02b0e3-dc52-4001-a78c-603754bc4453\" (UID: \"3a02b0e3-dc52-4001-a78c-603754bc4453\") "
Jul 27 19:40:42.283776 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.283708    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnkdm\" (UniqueName: \"kubernetes.io/projected/3a02b0e3-dc52-4001-a78c-603754bc4453-kube-api-access-dnkdm\") pod \"3a02b0e3-dc52-4001-a78c-603754bc4453\" (UID: \"3a02b0e3-dc52-4001-a78c-603754bc4453\") "
Jul 27 19:40:42.285641 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.285617    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a02b0e3-dc52-4001-a78c-603754bc4453-kube-api-access-dnkdm" (OuterVolumeSpecName: "kube-api-access-dnkdm") pod "3a02b0e3-dc52-4001-a78c-603754bc4453" (UID: "3a02b0e3-dc52-4001-a78c-603754bc4453"). InnerVolumeSpecName "kube-api-access-dnkdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:40:42.293926 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.293905    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a02b0e3-dc52-4001-a78c-603754bc4453-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3a02b0e3-dc52-4001-a78c-603754bc4453" (UID: "3a02b0e3-dc52-4001-a78c-603754bc4453"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:40:42.384817 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.384787    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a02b0e3-dc52-4001-a78c-603754bc4453-tls-cert\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:40:42.384817 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.384812    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnkdm\" (UniqueName: \"kubernetes.io/projected/3a02b0e3-dc52-4001-a78c-603754bc4453-kube-api-access-dnkdm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:40:42.386920 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.386894    2579 generic.go:356] "Generic (PLEG): container finished" podID="3a02b0e3-dc52-4001-a78c-603754bc4453" containerID="2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18" exitCode=0
Jul 27 19:40:42.387039 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.386936    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x"
Jul 27 19:40:42.387039 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.386940    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x" event={"ID":"3a02b0e3-dc52-4001-a78c-603754bc4453","Type":"ContainerDied","Data":"2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18"}
Jul 27 19:40:42.387039 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.386986    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ddbb5c7cf-22c4x" event={"ID":"3a02b0e3-dc52-4001-a78c-603754bc4453","Type":"ContainerDied","Data":"2b7e2beee7adea01fb1035534f514516411a8917e8a7e59c5ac5650f1ff252bf"}
Jul 27 19:40:42.387039 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.387005    2579 scope.go:117] "RemoveContainer" containerID="2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18"
Jul 27 19:40:42.396176 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.396162    2579 scope.go:117] "RemoveContainer" containerID="2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18"
Jul 27 19:40:42.396576 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:40:42.396555    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18\": container with ID starting with 2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18 not found: ID does not exist" containerID="2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18"
Jul 27 19:40:42.396631 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.396585    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18"} err="failed to get container status \"2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18\": rpc error: code = NotFound desc = could not find container \"2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18\": container with ID starting with 2a1b2053cc24471bac3db436442997b2e9a6e8d24aee68f5a7feb08b5f83cc18 not found: ID does not exist"
Jul 27 19:40:42.418946 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.418917    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ddbb5c7cf-22c4x"]
Jul 27 19:40:42.424693 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.424670    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5ddbb5c7cf-22c4x"]
Jul 27 19:40:42.489139 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:42.489071    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a02b0e3-dc52-4001-a78c-603754bc4453" path="/var/lib/kubelet/pods/3a02b0e3-dc52-4001-a78c-603754bc4453/volumes"
Jul 27 19:40:48.361904 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:48.361873    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:40:48.362975 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:48.362955    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:40:48.363183 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:48.363169    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:40:48.364289 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:40:48.364274    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:41:29.079817 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.079783    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5744b65d6-fwxnr"]
Jul 27 19:41:29.080462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.080009    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5744b65d6-fwxnr" podUID="e424a6b4-96b1-4bdc-8fcb-2d0c2a224267" containerName="manager" containerID="cri-o://01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc" gracePeriod=30
Jul 27 19:41:29.326975 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.326950    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:41:29.400330 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.400231    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dnjk\" (UniqueName: \"kubernetes.io/projected/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-kube-api-access-7dnjk\") pod \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") "
Jul 27 19:41:29.400330 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.400328    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert\") pod \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\" (UID: \"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267\") "
Jul 27 19:41:29.402352 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.402324    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e424a6b4-96b1-4bdc-8fcb-2d0c2a224267" (UID: "e424a6b4-96b1-4bdc-8fcb-2d0c2a224267"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:41:29.402481 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.402387    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-kube-api-access-7dnjk" (OuterVolumeSpecName: "kube-api-access-7dnjk") pod "e424a6b4-96b1-4bdc-8fcb-2d0c2a224267" (UID: "e424a6b4-96b1-4bdc-8fcb-2d0c2a224267"). InnerVolumeSpecName "kube-api-access-7dnjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:41:29.502576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.502446    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dnjk\" (UniqueName: \"kubernetes.io/projected/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-kube-api-access-7dnjk\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:41:29.502576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.502477    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267-webhook-cert\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:41:29.612300 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.612271    2579 generic.go:356] "Generic (PLEG): container finished" podID="e424a6b4-96b1-4bdc-8fcb-2d0c2a224267" containerID="01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc" exitCode=0
Jul 27 19:41:29.612464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.612315    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-fwxnr"
Jul 27 19:41:29.612464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.612333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-fwxnr" event={"ID":"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267","Type":"ContainerDied","Data":"01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc"}
Jul 27 19:41:29.612464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.612374    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-fwxnr" event={"ID":"e424a6b4-96b1-4bdc-8fcb-2d0c2a224267","Type":"ContainerDied","Data":"cd0222a8b332202dacd0834e42a611fbc0ec5ccb8b458623a29d0d0a780e3544"}
Jul 27 19:41:29.612464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.612396    2579 scope.go:117] "RemoveContainer" containerID="01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc"
Jul 27 19:41:29.621464 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.621447    2579 scope.go:117] "RemoveContainer" containerID="01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc"
Jul 27 19:41:29.621702 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:41:29.621683    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc\": container with ID starting with 01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc not found: ID does not exist" containerID="01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc"
Jul 27 19:41:29.621749 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.621711    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc"} err="failed to get container status \"01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc\": rpc error: code = NotFound desc = could not find container \"01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc\": container with ID starting with 01d427df94c398bcfffff378172caa03ff2676492e359bdb34383360b0ac1abc not found: ID does not exist"
Jul 27 19:41:29.632431 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.632408    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5744b65d6-fwxnr"]
Jul 27 19:41:29.635687 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:29.635665    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5744b65d6-fwxnr"]
Jul 27 19:41:30.488863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:41:30.488833    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e424a6b4-96b1-4bdc-8fcb-2d0c2a224267" path="/var/lib/kubelet/pods/e424a6b4-96b1-4bdc-8fcb-2d0c2a224267/volumes"
Jul 27 19:42:05.470603 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:05.470570    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:07.263331 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:07.263295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:24.146011 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:24.145910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:30.417819 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.417783    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5744b65d6-h2bln"]
Jul 27 19:42:30.418220 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.418146    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e424a6b4-96b1-4bdc-8fcb-2d0c2a224267" containerName="manager"
Jul 27 19:42:30.418220 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.418157    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e424a6b4-96b1-4bdc-8fcb-2d0c2a224267" containerName="manager"
Jul 27 19:42:30.418220 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.418167    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a02b0e3-dc52-4001-a78c-603754bc4453" containerName="authorino"
Jul 27 19:42:30.418220 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.418172    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a02b0e3-dc52-4001-a78c-603754bc4453" containerName="authorino"
Jul 27 19:42:30.418419 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.418261    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e424a6b4-96b1-4bdc-8fcb-2d0c2a224267" containerName="manager"
Jul 27 19:42:30.418419 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.418271    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a02b0e3-dc52-4001-a78c-603754bc4453" containerName="authorino"
Jul 27 19:42:30.421177 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.421162    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:30.423848 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.423827    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 19:42:30.425215 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.425197    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jcp46\""
Jul 27 19:42:30.425338 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.425286    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 19:42:30.430487 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.430463    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5744b65d6-h2bln"]
Jul 27 19:42:30.473571 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.473534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2992\" (UniqueName: \"kubernetes.io/projected/8ae337bf-35f4-46cf-a407-3b35cc7623b6-kube-api-access-w2992\") pod \"maas-controller-5744b65d6-h2bln\" (UID: \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\") " pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:30.473571 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.473572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae337bf-35f4-46cf-a407-3b35cc7623b6-webhook-cert\") pod \"maas-controller-5744b65d6-h2bln\" (UID: \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\") " pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:30.574980 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.574946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2992\" (UniqueName: \"kubernetes.io/projected/8ae337bf-35f4-46cf-a407-3b35cc7623b6-kube-api-access-w2992\") pod \"maas-controller-5744b65d6-h2bln\" (UID: \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\") " pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:30.574980 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.574986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae337bf-35f4-46cf-a407-3b35cc7623b6-webhook-cert\") pod \"maas-controller-5744b65d6-h2bln\" (UID: \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\") " pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:30.577380 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.577359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae337bf-35f4-46cf-a407-3b35cc7623b6-webhook-cert\") pod \"maas-controller-5744b65d6-h2bln\" (UID: \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\") " pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:30.583467 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.583446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2992\" (UniqueName: \"kubernetes.io/projected/8ae337bf-35f4-46cf-a407-3b35cc7623b6-kube-api-access-w2992\") pod \"maas-controller-5744b65d6-h2bln\" (UID: \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\") " pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:30.732471 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.732438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:30.862646 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.862625    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5744b65d6-h2bln"]
Jul 27 19:42:30.864522 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:42:30.864492    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae337bf_35f4_46cf_a407_3b35cc7623b6.slice/crio-9d82b06b70b3bbd1d09e39f78c4231e1bd29b609a090270fac29e1a1357df15a WatchSource:0}: Error finding container 9d82b06b70b3bbd1d09e39f78c4231e1bd29b609a090270fac29e1a1357df15a: Status 404 returned error can't find the container with id 9d82b06b70b3bbd1d09e39f78c4231e1bd29b609a090270fac29e1a1357df15a
Jul 27 19:42:30.910647 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:30.910614    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-h2bln" event={"ID":"8ae337bf-35f4-46cf-a407-3b35cc7623b6","Type":"ContainerStarted","Data":"9d82b06b70b3bbd1d09e39f78c4231e1bd29b609a090270fac29e1a1357df15a"}
Jul 27 19:42:31.917133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:31.917045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-h2bln" event={"ID":"8ae337bf-35f4-46cf-a407-3b35cc7623b6","Type":"ContainerStarted","Data":"5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f"}
Jul 27 19:42:31.917533 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:31.917171    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:31.935763 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:31.935715    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5744b65d6-h2bln" podStartSLOduration=1.366173031 podStartE2EDuration="1.935699929s" totalImagesPullingTime="569.526898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:42:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:42:31.933888766 +0000 UTC m=+1004.025240297" watchObservedRunningTime="2026-07-27 19:42:31.935699929 +0000 UTC m=+1004.027051492"
Jul 27 19:42:42.925981 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:42.925941    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:47.470563 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.470532    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:47.555747 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.555716    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5744b65d6-h2bln"]
Jul 27 19:42:47.555989 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.555962    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5744b65d6-h2bln" podUID="8ae337bf-35f4-46cf-a407-3b35cc7623b6" containerName="manager" containerID="cri-o://5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f" gracePeriod=30
Jul 27 19:42:47.797710 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.797682    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:47.939485 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.939450    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae337bf-35f4-46cf-a407-3b35cc7623b6-webhook-cert\") pod \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\" (UID: \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\") "
Jul 27 19:42:47.939671 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.939509    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2992\" (UniqueName: \"kubernetes.io/projected/8ae337bf-35f4-46cf-a407-3b35cc7623b6-kube-api-access-w2992\") pod \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\" (UID: \"8ae337bf-35f4-46cf-a407-3b35cc7623b6\") "
Jul 27 19:42:47.941416 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.941390    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ae337bf-35f4-46cf-a407-3b35cc7623b6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8ae337bf-35f4-46cf-a407-3b35cc7623b6" (UID: "8ae337bf-35f4-46cf-a407-3b35cc7623b6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:42:47.941559 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.941531    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ae337bf-35f4-46cf-a407-3b35cc7623b6-kube-api-access-w2992" (OuterVolumeSpecName: "kube-api-access-w2992") pod "8ae337bf-35f4-46cf-a407-3b35cc7623b6" (UID: "8ae337bf-35f4-46cf-a407-3b35cc7623b6"). InnerVolumeSpecName "kube-api-access-w2992". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:42:47.996575 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.996496    2579 generic.go:356] "Generic (PLEG): container finished" podID="8ae337bf-35f4-46cf-a407-3b35cc7623b6" containerID="5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f" exitCode=0
Jul 27 19:42:47.996575 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.996544    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-h2bln"
Jul 27 19:42:47.996785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.996588    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-h2bln" event={"ID":"8ae337bf-35f4-46cf-a407-3b35cc7623b6","Type":"ContainerDied","Data":"5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f"}
Jul 27 19:42:47.996785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.996643    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-h2bln" event={"ID":"8ae337bf-35f4-46cf-a407-3b35cc7623b6","Type":"ContainerDied","Data":"9d82b06b70b3bbd1d09e39f78c4231e1bd29b609a090270fac29e1a1357df15a"}
Jul 27 19:42:47.996785 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:47.996674    2579 scope.go:117] "RemoveContainer" containerID="5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f"
Jul 27 19:42:48.005506 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.005490    2579 scope.go:117] "RemoveContainer" containerID="5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f"
Jul 27 19:42:48.005771 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:42:48.005751    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f\": container with ID starting with 5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f not found: ID does not exist" containerID="5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f"
Jul 27 19:42:48.005818 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.005779    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f"} err="failed to get container status \"5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f\": rpc error: code = NotFound desc = could not find container \"5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f\": container with ID starting with 5266fe9938da488bbefea18e704fe45ac4ff121de0975eb201188442be1d887f not found: ID does not exist"
Jul 27 19:42:48.016707 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.016686    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5744b65d6-h2bln"]
Jul 27 19:42:48.020386 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.020364    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5744b65d6-h2bln"]
Jul 27 19:42:48.040675 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.040653    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae337bf-35f4-46cf-a407-3b35cc7623b6-webhook-cert\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:42:48.040763 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.040682    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2992\" (UniqueName: \"kubernetes.io/projected/8ae337bf-35f4-46cf-a407-3b35cc7623b6-kube-api-access-w2992\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:42:48.488497 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.488467    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ae337bf-35f4-46cf-a407-3b35cc7623b6" path="/var/lib/kubelet/pods/8ae337bf-35f4-46cf-a407-3b35cc7623b6/volumes"
Jul 27 19:42:48.700470 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.700443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:42:48.803796 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.803766    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5744b65d6-pfq5n"]
Jul 27 19:42:48.804229 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.804212    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ae337bf-35f4-46cf-a407-3b35cc7623b6" containerName="manager"
Jul 27 19:42:48.804340 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.804232    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ae337bf-35f4-46cf-a407-3b35cc7623b6" containerName="manager"
Jul 27 19:42:48.804399 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.804375    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ae337bf-35f4-46cf-a407-3b35cc7623b6" containerName="manager"
Jul 27 19:42:48.808957 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.808938    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:48.812400 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.812384    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 19:42:48.812511 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.812387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jcp46\""
Jul 27 19:42:48.812511 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.812387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 19:42:48.815624 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.815598    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5744b65d6-pfq5n"]
Jul 27 19:42:48.949772 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.949739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f49e78b-59ed-490e-bfe7-535443db2fbf-webhook-cert\") pod \"maas-controller-5744b65d6-pfq5n\" (UID: \"2f49e78b-59ed-490e-bfe7-535443db2fbf\") " pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:48.949772 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:48.949775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kps4\" (UniqueName: \"kubernetes.io/projected/2f49e78b-59ed-490e-bfe7-535443db2fbf-kube-api-access-2kps4\") pod \"maas-controller-5744b65d6-pfq5n\" (UID: \"2f49e78b-59ed-490e-bfe7-535443db2fbf\") " pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:49.051047 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:49.050951    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f49e78b-59ed-490e-bfe7-535443db2fbf-webhook-cert\") pod \"maas-controller-5744b65d6-pfq5n\" (UID: \"2f49e78b-59ed-490e-bfe7-535443db2fbf\") " pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:49.051047 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:49.050992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kps4\" (UniqueName: \"kubernetes.io/projected/2f49e78b-59ed-490e-bfe7-535443db2fbf-kube-api-access-2kps4\") pod \"maas-controller-5744b65d6-pfq5n\" (UID: \"2f49e78b-59ed-490e-bfe7-535443db2fbf\") " pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:49.053482 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:49.053456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f49e78b-59ed-490e-bfe7-535443db2fbf-webhook-cert\") pod \"maas-controller-5744b65d6-pfq5n\" (UID: \"2f49e78b-59ed-490e-bfe7-535443db2fbf\") " pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:49.059111 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:49.059093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kps4\" (UniqueName: \"kubernetes.io/projected/2f49e78b-59ed-490e-bfe7-535443db2fbf-kube-api-access-2kps4\") pod \"maas-controller-5744b65d6-pfq5n\" (UID: \"2f49e78b-59ed-490e-bfe7-535443db2fbf\") " pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:49.121099 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:49.121068    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:49.250264 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:49.250226    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5744b65d6-pfq5n"]
Jul 27 19:42:49.252734 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:42:49.252706    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f49e78b_59ed_490e_bfe7_535443db2fbf.slice/crio-2cea03112dfd16e518dbcc231ec2f34aaaa776f662a16e36181f99f3a6b445a8 WatchSource:0}: Error finding container 2cea03112dfd16e518dbcc231ec2f34aaaa776f662a16e36181f99f3a6b445a8: Status 404 returned error can't find the container with id 2cea03112dfd16e518dbcc231ec2f34aaaa776f662a16e36181f99f3a6b445a8
Jul 27 19:42:50.007485 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:50.007452    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-pfq5n" event={"ID":"2f49e78b-59ed-490e-bfe7-535443db2fbf","Type":"ContainerStarted","Data":"7076a2fee3cb3b8e750e71d374a16969fbe0b34e1ad3729f976a1d3764bcb820"}
Jul 27 19:42:50.007926 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:50.007503    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744b65d6-pfq5n" event={"ID":"2f49e78b-59ed-490e-bfe7-535443db2fbf","Type":"ContainerStarted","Data":"2cea03112dfd16e518dbcc231ec2f34aaaa776f662a16e36181f99f3a6b445a8"}
Jul 27 19:42:50.007926 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:50.007587    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:42:50.025041 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:42:50.024991    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5744b65d6-pfq5n" podStartSLOduration=1.681649015 podStartE2EDuration="2.024975286s" totalImagesPullingTime="343.326271ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:42:50.02359799 +0000 UTC m=+1022.114949488" watchObservedRunningTime="2026-07-27 19:42:50.024975286 +0000 UTC m=+1022.116326784"
Jul 27 19:43:01.014333 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:01.014287    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5744b65d6-pfq5n"
Jul 27 19:43:16.844391 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:16.844353    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:16.944205 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:16.944174    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:17.066526 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:17.066492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:22.950564 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:22.950532    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:23.060877 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:23.060835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:23.160687 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:23.160644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:23.291408 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:23.291320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:24.202524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:24.202474    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:29.054555 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:29.054521    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:43:29.122885 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:43:29.122849    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:45:00.145513 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:00.145481    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct"]
Jul 27 19:45:00.148871 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:00.148853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct"
Jul 27 19:45:00.151483 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:00.151462    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 19:45:00.158202 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:00.158183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct"
Jul 27 19:45:00.162374 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:00.162352    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct"]
Jul 27 19:45:00.302504 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:00.302424    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct"]
Jul 27 19:45:00.305042 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:45:00.305017    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282d2290_ab83_495b_973f_6092173a7aa8.slice/crio-3ed8d6f82f3a730caa111868d200b566014b1d5ecca5c696a4a013a9a5e480b5 WatchSource:0}: Error finding container 3ed8d6f82f3a730caa111868d200b566014b1d5ecca5c696a4a013a9a5e480b5: Status 404 returned error can't find the container with id 3ed8d6f82f3a730caa111868d200b566014b1d5ecca5c696a4a013a9a5e480b5
Jul 27 19:45:00.307282 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:00.307268    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:45:00.631979 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:00.631944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct" event={"ID":"282d2290-ab83-495b-973f-6092173a7aa8","Type":"ContainerStarted","Data":"3ed8d6f82f3a730caa111868d200b566014b1d5ecca5c696a4a013a9a5e480b5"}
Jul 27 19:45:03.649171 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:03.649133    2579 generic.go:356] "Generic (PLEG): container finished" podID="282d2290-ab83-495b-973f-6092173a7aa8" containerID="9c2f7cfa4428e457dcdad26589948825efd8ff5dcc17fb94efee34618f70e617" exitCode=0
Jul 27 19:45:03.649583 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:03.649214    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct" event={"ID":"282d2290-ab83-495b-973f-6092173a7aa8","Type":"ContainerDied","Data":"9c2f7cfa4428e457dcdad26589948825efd8ff5dcc17fb94efee34618f70e617"}
Jul 27 19:45:04.778524 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:04.778500    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct"
Jul 27 19:45:05.661766 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:05.661740    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct"
Jul 27 19:45:05.662026 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:05.661773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753025-f5gct" event={"ID":"282d2290-ab83-495b-973f-6092173a7aa8","Type":"ContainerDied","Data":"3ed8d6f82f3a730caa111868d200b566014b1d5ecca5c696a4a013a9a5e480b5"}
Jul 27 19:45:05.662026 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:05.661831    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ed8d6f82f3a730caa111868d200b566014b1d5ecca5c696a4a013a9a5e480b5"
Jul 27 19:45:48.393644 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:48.393615    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:45:48.394704 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:48.394687    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:45:48.394820 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:48.394771    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:45:48.395831 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:48.395812    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:45:58.215439 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:45:58.215404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:46:21.112811 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:46:21.112779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:46:34.051456 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:46:34.051424    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:46:34.142116 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:46:34.142073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:47:04.843745 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:47:04.843713    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:47:04.943907 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:47:04.943877    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:47:25.988916 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:47:25.988878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:47:27.634812 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:47:27.634772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:50:35.085046 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:50:35.085013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:50:48.424960 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:50:48.424930    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:50:48.425587 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:50:48.425569    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:50:48.426085 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:50:48.426062    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:50:48.426719 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:50:48.426700    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:51:11.810296 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:11.810232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:51:22.679752 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.679716    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"]
Jul 27 19:51:22.680350 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.680333    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="282d2290-ab83-495b-973f-6092173a7aa8" containerName="cleanup"
Jul 27 19:51:22.680404 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.680353    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="282d2290-ab83-495b-973f-6092173a7aa8" containerName="cleanup"
Jul 27 19:51:22.680475 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.680462    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="282d2290-ab83-495b-973f-6092173a7aa8" containerName="cleanup"
Jul 27 19:51:22.683737 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.683717    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.686091 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.686072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f3c3fead-kserve-self-signed-certs\""
Jul 27 19:51:22.690662 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.690638    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"]
Jul 27 19:51:22.713756 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.713727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:51:22.774853 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.774815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tmp-dir\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.775028 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.774868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-model-cache\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.775028 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.774885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-dshm\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.775028 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.774899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tls-certs\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.775028 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.774959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjl5h\" (UniqueName: \"kubernetes.io/projected/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-kube-api-access-qjl5h\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.775221 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.775090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-home\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876196 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tmp-dir\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876424 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876209    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-model-cache\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876424 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876227    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-dshm\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876424 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tls-certs\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876424 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjl5h\" (UniqueName: \"kubernetes.io/projected/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-kube-api-access-qjl5h\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876424 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-home\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876703 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tmp-dir\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876703 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876647    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-model-cache\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.876774 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.876706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-home\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.878511 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.878489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-dshm\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.878699 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.878681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tls-certs\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.889454 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.889422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjl5h\" (UniqueName: \"kubernetes.io/projected/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-kube-api-access-qjl5h\") pod \"e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") " pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:22.995381 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:22.995305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:23.137071 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:23.137048    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"]
Jul 27 19:51:23.139446 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:51:23.139420    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1da862c8_ef2f_470f_b51e_c1bf8884d5dc.slice/crio-b0351a185676693a6e546ffd8e51ef57b463762face1c19df0d4570f2ee6eedb WatchSource:0}: Error finding container b0351a185676693a6e546ffd8e51ef57b463762face1c19df0d4570f2ee6eedb: Status 404 returned error can't find the container with id b0351a185676693a6e546ffd8e51ef57b463762face1c19df0d4570f2ee6eedb
Jul 27 19:51:23.469307 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:23.469272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z" event={"ID":"1da862c8-ef2f-470f-b51e-c1bf8884d5dc","Type":"ContainerStarted","Data":"c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467"}
Jul 27 19:51:23.469488 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:23.469320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z" event={"ID":"1da862c8-ef2f-470f-b51e-c1bf8884d5dc","Type":"ContainerStarted","Data":"b0351a185676693a6e546ffd8e51ef57b463762face1c19df0d4570f2ee6eedb"}
Jul 27 19:51:32.996263 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:32.996202    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:32.996263 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:32.996269    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:33.009039 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:33.009013    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:33.027921 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:33.027873    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z" podStartSLOduration=11.027857676 podStartE2EDuration="11.027857676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:51:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:51:23.486586018 +0000 UTC m=+1535.577937513" watchObservedRunningTime="2026-07-27 19:51:33.027857676 +0000 UTC m=+1545.119209173"
Jul 27 19:51:33.530369 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:33.530343    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:51:48.764420 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:48.764388    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:51:50.222418 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:50.222383    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"]
Jul 27 19:51:50.240424 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:50.240396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:51:53.089443 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:51:53.089410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:52:05.245076 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.245025    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z" podUID="1da862c8-ef2f-470f-b51e-c1bf8884d5dc" containerName="main" containerID="cri-o://c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467" gracePeriod=45
Jul 27 19:52:05.496404 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.496346    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:52:05.588860 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.588825    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-home\") pod \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") "
Jul 27 19:52:05.588860 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.588863    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tmp-dir\") pod \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") "
Jul 27 19:52:05.589122 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.588891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tls-certs\") pod \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") "
Jul 27 19:52:05.589122 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.588992    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-dshm\") pod \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") "
Jul 27 19:52:05.589122 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.589026    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjl5h\" (UniqueName: \"kubernetes.io/projected/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-kube-api-access-qjl5h\") pod \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") "
Jul 27 19:52:05.589122 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.589066    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-model-cache\") pod \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\" (UID: \"1da862c8-ef2f-470f-b51e-c1bf8884d5dc\") "
Jul 27 19:52:05.589374 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.589163    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1da862c8-ef2f-470f-b51e-c1bf8884d5dc" (UID: "1da862c8-ef2f-470f-b51e-c1bf8884d5dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:52:05.589374 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.589173    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-home" (OuterVolumeSpecName: "home") pod "1da862c8-ef2f-470f-b51e-c1bf8884d5dc" (UID: "1da862c8-ef2f-470f-b51e-c1bf8884d5dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:52:05.589495 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.589374    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "1da862c8-ef2f-470f-b51e-c1bf8884d5dc" (UID: "1da862c8-ef2f-470f-b51e-c1bf8884d5dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:52:05.589552 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.589531    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-model-cache\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:52:05.589552 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.589549    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-home\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:52:05.589661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.589584    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tmp-dir\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:52:05.591133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.591105    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-dshm" (OuterVolumeSpecName: "dshm") pod "1da862c8-ef2f-470f-b51e-c1bf8884d5dc" (UID: "1da862c8-ef2f-470f-b51e-c1bf8884d5dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:52:05.591269 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.591209    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1da862c8-ef2f-470f-b51e-c1bf8884d5dc" (UID: "1da862c8-ef2f-470f-b51e-c1bf8884d5dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:52:05.591316 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.591292    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-kube-api-access-qjl5h" (OuterVolumeSpecName: "kube-api-access-qjl5h") pod "1da862c8-ef2f-470f-b51e-c1bf8884d5dc" (UID: "1da862c8-ef2f-470f-b51e-c1bf8884d5dc"). InnerVolumeSpecName "kube-api-access-qjl5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:52:05.685663 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.685629    2579 generic.go:356] "Generic (PLEG): container finished" podID="1da862c8-ef2f-470f-b51e-c1bf8884d5dc" containerID="c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467" exitCode=0
Jul 27 19:52:05.685852 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.685670    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z" event={"ID":"1da862c8-ef2f-470f-b51e-c1bf8884d5dc","Type":"ContainerDied","Data":"c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467"}
Jul 27 19:52:05.685852 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.685679    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"
Jul 27 19:52:05.685852 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.685703    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z" event={"ID":"1da862c8-ef2f-470f-b51e-c1bf8884d5dc","Type":"ContainerDied","Data":"b0351a185676693a6e546ffd8e51ef57b463762face1c19df0d4570f2ee6eedb"}
Jul 27 19:52:05.685852 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.685720    2579 scope.go:117] "RemoveContainer" containerID="c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467"
Jul 27 19:52:05.690140 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.690117    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-tls-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:52:05.690140 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.690140    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-dshm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:52:05.690364 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.690149    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjl5h\" (UniqueName: \"kubernetes.io/projected/1da862c8-ef2f-470f-b51e-c1bf8884d5dc-kube-api-access-qjl5h\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:52:05.697016 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.696993    2579 scope.go:117] "RemoveContainer" containerID="c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467"
Jul 27 19:52:05.697581 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:52:05.697557    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467\": container with ID starting with c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467 not found: ID does not exist" containerID="c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467"
Jul 27 19:52:05.697678 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.697594    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467"} err="failed to get container status \"c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467\": rpc error: code = NotFound desc = could not find container \"c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467\": container with ID starting with c8b01d7379070742f990a2b916b21f2a5ec215e55705829868121fcab5f2d467 not found: ID does not exist"
Jul 27 19:52:05.707360 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.707336    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"]
Jul 27 19:52:05.710690 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:05.710665    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f3c3fead-kserve-686cf6647-khz7z"]
Jul 27 19:52:06.488763 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:52:06.488720    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da862c8-ef2f-470f-b51e-c1bf8884d5dc" path="/var/lib/kubelet/pods/1da862c8-ef2f-470f-b51e-c1bf8884d5dc/volumes"
Jul 27 19:54:26.411462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:54:26.411372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:05.737918 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:05.737861    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:31.213179 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:31.213145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:34.209571 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:34.209542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:34.307372 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:34.307339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:34.643515 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:34.643433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:40.328576 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:40.328498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:40.451034 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:40.451000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:40.595438 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:40.595356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:40.875283 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:40.875185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:41.143619 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:41.143549    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:42.870112 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:42.870075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:42.974942 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:42.974912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:43.134330 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.134258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:43.235469 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.235434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:43.577145 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.577112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:43.677624 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.677596    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:55:43.704319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.704289    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"]
Jul 27 19:55:43.705078 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.705056    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da862c8-ef2f-470f-b51e-c1bf8884d5dc" containerName="main"
Jul 27 19:55:43.705265 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.705229    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da862c8-ef2f-470f-b51e-c1bf8884d5dc" containerName="main"
Jul 27 19:55:43.705380 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.705356    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da862c8-ef2f-470f-b51e-c1bf8884d5dc" containerName="main"
Jul 27 19:55:43.709018 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.708682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:43.711335 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.711313    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-6154550c-dockercfg-z7nft\""
Jul 27 19:55:43.711539 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.711439    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-6154550c\""
Jul 27 19:55:43.718438 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.718417    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"]
Jul 27 19:55:43.726559 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.726538    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"]
Jul 27 19:55:43.730547 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.730529    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:43.740839 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.740812    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"]
Jul 27 19:55:43.743817 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.743794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6fcs\" (UniqueName: \"kubernetes.io/projected/34ad9193-2191-49f2-a565-62fca517e93e-kube-api-access-k6fcs\") pod \"payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w\" (UID: \"34ad9193-2191-49f2-a565-62fca517e93e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:43.743920 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.743836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad9193-2191-49f2-a565-62fca517e93e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w\" (UID: \"34ad9193-2191-49f2-a565-62fca517e93e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:43.844705 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.844615    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad9193-2191-49f2-a565-62fca517e93e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w\" (UID: \"34ad9193-2191-49f2-a565-62fca517e93e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:43.844875 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.844739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9ll5\" (UniqueName: \"kubernetes.io/projected/46ab011e-dac6-4f34-8c47-7357fb21b24e-kube-api-access-c9ll5\") pod \"payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs\" (UID: \"46ab011e-dac6-4f34-8c47-7357fb21b24e\") " pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:43.844875 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.844764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ab011e-dac6-4f34-8c47-7357fb21b24e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs\" (UID: \"46ab011e-dac6-4f34-8c47-7357fb21b24e\") " pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:43.844875 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.844784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6fcs\" (UniqueName: \"kubernetes.io/projected/34ad9193-2191-49f2-a565-62fca517e93e-kube-api-access-k6fcs\") pod \"payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w\" (UID: \"34ad9193-2191-49f2-a565-62fca517e93e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:43.845436 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.845418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad9193-2191-49f2-a565-62fca517e93e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w\" (UID: \"34ad9193-2191-49f2-a565-62fca517e93e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:43.852537 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.852504    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6fcs\" (UniqueName: \"kubernetes.io/projected/34ad9193-2191-49f2-a565-62fca517e93e-kube-api-access-k6fcs\") pod \"payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w\" (UID: \"34ad9193-2191-49f2-a565-62fca517e93e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:43.946052 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.946022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9ll5\" (UniqueName: \"kubernetes.io/projected/46ab011e-dac6-4f34-8c47-7357fb21b24e-kube-api-access-c9ll5\") pod \"payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs\" (UID: \"46ab011e-dac6-4f34-8c47-7357fb21b24e\") " pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:43.946052 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.946056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ab011e-dac6-4f34-8c47-7357fb21b24e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs\" (UID: \"46ab011e-dac6-4f34-8c47-7357fb21b24e\") " pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:43.946712 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.946694    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ab011e-dac6-4f34-8c47-7357fb21b24e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs\" (UID: \"46ab011e-dac6-4f34-8c47-7357fb21b24e\") " pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:43.953672 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:43.953637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9ll5\" (UniqueName: \"kubernetes.io/projected/46ab011e-dac6-4f34-8c47-7357fb21b24e-kube-api-access-c9ll5\") pod \"payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs\" (UID: \"46ab011e-dac6-4f34-8c47-7357fb21b24e\") " pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:44.023575 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.023544    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:44.042864 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.042836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:44.168151 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.168118    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"]
Jul 27 19:55:44.170626 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:55:44.170584    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ad9193_2191_49f2_a565_62fca517e93e.slice/crio-393df46b54fefb7404e5dfcf429905b770363b9e6390973f627f3888f05628e0 WatchSource:0}: Error finding container 393df46b54fefb7404e5dfcf429905b770363b9e6390973f627f3888f05628e0: Status 404 returned error can't find the container with id 393df46b54fefb7404e5dfcf429905b770363b9e6390973f627f3888f05628e0
Jul 27 19:55:44.198368 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.198346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"]
Jul 27 19:55:44.201396 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:55:44.201372    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ab011e_dac6_4f34_8c47_7357fb21b24e.slice/crio-3521de863460e95e9ca19891aab0aca2e3957efeabd5d6516b79c04075c16704 WatchSource:0}: Error finding container 3521de863460e95e9ca19891aab0aca2e3957efeabd5d6516b79c04075c16704: Status 404 returned error can't find the container with id 3521de863460e95e9ca19891aab0aca2e3957efeabd5d6516b79c04075c16704
Jul 27 19:55:44.750284 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.750215    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs" event={"ID":"46ab011e-dac6-4f34-8c47-7357fb21b24e","Type":"ContainerStarted","Data":"b2eda934345bfefcf8d253628f259a66d87be2fdc45d84e620b98ee43880cb80"}
Jul 27 19:55:44.750462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.750303    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs" event={"ID":"46ab011e-dac6-4f34-8c47-7357fb21b24e","Type":"ContainerStarted","Data":"3521de863460e95e9ca19891aab0aca2e3957efeabd5d6516b79c04075c16704"}
Jul 27 19:55:44.750462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.750337    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:55:44.751992 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.751963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w" event={"ID":"34ad9193-2191-49f2-a565-62fca517e93e","Type":"ContainerStarted","Data":"6f97079485170dca2779fb58da042bbfaf53fd6b25ea5961bf0c3d5091d5b1c4"}
Jul 27 19:55:44.752132 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.752012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w" event={"ID":"34ad9193-2191-49f2-a565-62fca517e93e","Type":"ContainerStarted","Data":"393df46b54fefb7404e5dfcf429905b770363b9e6390973f627f3888f05628e0"}
Jul 27 19:55:44.752132 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.752042    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:55:44.766956 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.766915    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs" podStartSLOduration=1.7668995619999999 podStartE2EDuration="1.766899562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:55:44.765304364 +0000 UTC m=+1796.856655863" watchObservedRunningTime="2026-07-27 19:55:44.766899562 +0000 UTC m=+1796.858251101"
Jul 27 19:55:44.782824 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:44.782776    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w" podStartSLOduration=1.78275812 podStartE2EDuration="1.78275812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:55:44.779882131 +0000 UTC m=+1796.871233630" watchObservedRunningTime="2026-07-27 19:55:44.78275812 +0000 UTC m=+1796.874109619"
Jul 27 19:55:48.458780 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:48.458744    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:55:48.459910 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:48.459890    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:55:48.460457 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:48.460435    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 19:55:48.461561 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:48.461537    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 19:55:55.759880 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:55.759826    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w" podUID="34ad9193-2191-49f2-a565-62fca517e93e" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.42:9004: connect: connection refused"
Jul 27 19:55:55.760341 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:55:55.760289    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:56:05.759732 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:05.759699    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:56:08.314847 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.314815    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"]
Jul 27 19:56:08.318652 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.318631    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"
Jul 27 19:56:08.321157 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.321138    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 19:56:08.324563 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.324542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"]
Jul 27 19:56:08.472199 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.472159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b73ed-c5f5-4962-83ef-c6101187eb21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6154550c-v85zs\" (UID: \"f06b73ed-c5f5-4962-83ef-c6101187eb21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"
Jul 27 19:56:08.573672 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.573571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b73ed-c5f5-4962-83ef-c6101187eb21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6154550c-v85zs\" (UID: \"f06b73ed-c5f5-4962-83ef-c6101187eb21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"
Jul 27 19:56:08.574382 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.574359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b73ed-c5f5-4962-83ef-c6101187eb21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6154550c-v85zs\" (UID: \"f06b73ed-c5f5-4962-83ef-c6101187eb21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"
Jul 27 19:56:08.630548 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.630516    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"
Jul 27 19:56:08.772297 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.768873    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"]
Jul 27 19:56:08.776589 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:56:08.776559    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf06b73ed_c5f5_4962_83ef_c6101187eb21.slice/crio-5139706bdf7a21fec6adb0ba0bd4cbc671d4f98c684dbcafc9fce7c1abe81c2a WatchSource:0}: Error finding container 5139706bdf7a21fec6adb0ba0bd4cbc671d4f98c684dbcafc9fce7c1abe81c2a: Status 404 returned error can't find the container with id 5139706bdf7a21fec6adb0ba0bd4cbc671d4f98c684dbcafc9fce7c1abe81c2a
Jul 27 19:56:08.876357 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:08.876273    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs" event={"ID":"f06b73ed-c5f5-4962-83ef-c6101187eb21","Type":"ContainerStarted","Data":"5139706bdf7a21fec6adb0ba0bd4cbc671d4f98c684dbcafc9fce7c1abe81c2a"}
Jul 27 19:56:09.884041 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:09.883941    2579 generic.go:356] "Generic (PLEG): container finished" podID="f06b73ed-c5f5-4962-83ef-c6101187eb21" containerID="4b5625973beae60bac42220c98b53b5ca1acaf1b0f9693926e738f0e0ea02c79" exitCode=0
Jul 27 19:56:09.884440 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:09.884028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs" event={"ID":"f06b73ed-c5f5-4962-83ef-c6101187eb21","Type":"ContainerDied","Data":"4b5625973beae60bac42220c98b53b5ca1acaf1b0f9693926e738f0e0ea02c79"}
Jul 27 19:56:11.023896 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:11.023873    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"
Jul 27 19:56:11.197659 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:11.197630    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b73ed-c5f5-4962-83ef-c6101187eb21-maas-api-service-ca\") pod \"f06b73ed-c5f5-4962-83ef-c6101187eb21\" (UID: \"f06b73ed-c5f5-4962-83ef-c6101187eb21\") "
Jul 27 19:56:11.197973 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:11.197947    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f06b73ed-c5f5-4962-83ef-c6101187eb21-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f06b73ed-c5f5-4962-83ef-c6101187eb21" (UID: "f06b73ed-c5f5-4962-83ef-c6101187eb21"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:56:11.298388 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:11.298357    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b73ed-c5f5-4962-83ef-c6101187eb21-maas-api-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:56:11.895435 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:11.895407    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"
Jul 27 19:56:11.895639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:11.895455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs" event={"ID":"f06b73ed-c5f5-4962-83ef-c6101187eb21","Type":"ContainerDied","Data":"5139706bdf7a21fec6adb0ba0bd4cbc671d4f98c684dbcafc9fce7c1abe81c2a"}
Jul 27 19:56:11.895639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:11.895513    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5139706bdf7a21fec6adb0ba0bd4cbc671d4f98c684dbcafc9fce7c1abe81c2a"
Jul 27 19:56:18.727455 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.727420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:18.750925 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.750889    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"]
Jul 27 19:56:18.751129 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.751101    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs" podUID="46ab011e-dac6-4f34-8c47-7357fb21b24e" containerName="payload-processing" containerID="cri-o://b2eda934345bfefcf8d253628f259a66d87be2fdc45d84e620b98ee43880cb80" gracePeriod=30
Jul 27 19:56:18.758308 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.758205    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"]
Jul 27 19:56:18.758463 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.758441    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w" podUID="34ad9193-2191-49f2-a565-62fca517e93e" containerName="payload-pre-processing" containerID="cri-o://6f97079485170dca2779fb58da042bbfaf53fd6b25ea5961bf0c3d5091d5b1c4" gracePeriod=30
Jul 27 19:56:18.827773 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.827742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:18.891443 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.891413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:18.932833 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.932801    2579 generic.go:356] "Generic (PLEG): container finished" podID="34ad9193-2191-49f2-a565-62fca517e93e" containerID="6f97079485170dca2779fb58da042bbfaf53fd6b25ea5961bf0c3d5091d5b1c4" exitCode=0
Jul 27 19:56:18.932988 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.932873    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w" event={"ID":"34ad9193-2191-49f2-a565-62fca517e93e","Type":"ContainerDied","Data":"6f97079485170dca2779fb58da042bbfaf53fd6b25ea5961bf0c3d5091d5b1c4"}
Jul 27 19:56:18.934775 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.934751    2579 generic.go:356] "Generic (PLEG): container finished" podID="46ab011e-dac6-4f34-8c47-7357fb21b24e" containerID="b2eda934345bfefcf8d253628f259a66d87be2fdc45d84e620b98ee43880cb80" exitCode=0
Jul 27 19:56:18.934923 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:18.934832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs" event={"ID":"46ab011e-dac6-4f34-8c47-7357fb21b24e","Type":"ContainerDied","Data":"b2eda934345bfefcf8d253628f259a66d87be2fdc45d84e620b98ee43880cb80"}
Jul 27 19:56:19.026321 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.026299    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:56:19.029197 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.029178    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:56:19.066918 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.066886    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9ll5\" (UniqueName: \"kubernetes.io/projected/46ab011e-dac6-4f34-8c47-7357fb21b24e-kube-api-access-c9ll5\") pod \"46ab011e-dac6-4f34-8c47-7357fb21b24e\" (UID: \"46ab011e-dac6-4f34-8c47-7357fb21b24e\") "
Jul 27 19:56:19.067098 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.066981    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6fcs\" (UniqueName: \"kubernetes.io/projected/34ad9193-2191-49f2-a565-62fca517e93e-kube-api-access-k6fcs\") pod \"34ad9193-2191-49f2-a565-62fca517e93e\" (UID: \"34ad9193-2191-49f2-a565-62fca517e93e\") "
Jul 27 19:56:19.067098 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.067072    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad9193-2191-49f2-a565-62fca517e93e-plugins-config-volume\") pod \"34ad9193-2191-49f2-a565-62fca517e93e\" (UID: \"34ad9193-2191-49f2-a565-62fca517e93e\") "
Jul 27 19:56:19.067223 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.067115    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ab011e-dac6-4f34-8c47-7357fb21b24e-plugins-config-volume\") pod \"46ab011e-dac6-4f34-8c47-7357fb21b24e\" (UID: \"46ab011e-dac6-4f34-8c47-7357fb21b24e\") "
Jul 27 19:56:19.067481 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.067451    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34ad9193-2191-49f2-a565-62fca517e93e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34ad9193-2191-49f2-a565-62fca517e93e" (UID: "34ad9193-2191-49f2-a565-62fca517e93e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:56:19.067634 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.067521    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46ab011e-dac6-4f34-8c47-7357fb21b24e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "46ab011e-dac6-4f34-8c47-7357fb21b24e" (UID: "46ab011e-dac6-4f34-8c47-7357fb21b24e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:56:19.069159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.069135    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46ab011e-dac6-4f34-8c47-7357fb21b24e-kube-api-access-c9ll5" (OuterVolumeSpecName: "kube-api-access-c9ll5") pod "46ab011e-dac6-4f34-8c47-7357fb21b24e" (UID: "46ab011e-dac6-4f34-8c47-7357fb21b24e"). InnerVolumeSpecName "kube-api-access-c9ll5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:56:19.069299 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.069173    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34ad9193-2191-49f2-a565-62fca517e93e-kube-api-access-k6fcs" (OuterVolumeSpecName: "kube-api-access-k6fcs") pod "34ad9193-2191-49f2-a565-62fca517e93e" (UID: "34ad9193-2191-49f2-a565-62fca517e93e"). InnerVolumeSpecName "kube-api-access-k6fcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:56:19.168098 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.168064    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad9193-2191-49f2-a565-62fca517e93e-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:56:19.168098 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.168089    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ab011e-dac6-4f34-8c47-7357fb21b24e-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:56:19.168319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.168120    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9ll5\" (UniqueName: \"kubernetes.io/projected/46ab011e-dac6-4f34-8c47-7357fb21b24e-kube-api-access-c9ll5\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:56:19.168319 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.168130    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6fcs\" (UniqueName: \"kubernetes.io/projected/34ad9193-2191-49f2-a565-62fca517e93e-kube-api-access-k6fcs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:56:19.945781 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.945749    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"
Jul 27 19:56:19.945781 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.945768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs" event={"ID":"46ab011e-dac6-4f34-8c47-7357fb21b24e","Type":"ContainerDied","Data":"3521de863460e95e9ca19891aab0aca2e3957efeabd5d6516b79c04075c16704"}
Jul 27 19:56:19.946308 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.945846    2579 scope.go:117] "RemoveContainer" containerID="b2eda934345bfefcf8d253628f259a66d87be2fdc45d84e620b98ee43880cb80"
Jul 27 19:56:19.947234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.947213    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"
Jul 27 19:56:19.947234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.947224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w" event={"ID":"34ad9193-2191-49f2-a565-62fca517e93e","Type":"ContainerDied","Data":"393df46b54fefb7404e5dfcf429905b770363b9e6390973f627f3888f05628e0"}
Jul 27 19:56:19.954441 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.954425    2579 scope.go:117] "RemoveContainer" containerID="6f97079485170dca2779fb58da042bbfaf53fd6b25ea5961bf0c3d5091d5b1c4"
Jul 27 19:56:19.970182 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.970159    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"]
Jul 27 19:56:19.975392 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.975370    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6154550c-7d86c74d4b-467rs"]
Jul 27 19:56:19.984611 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.984583    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"]
Jul 27 19:56:19.989910 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:19.989889    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6154550c-5cf845dcb4-l846w"]
Jul 27 19:56:20.488799 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:20.488764    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34ad9193-2191-49f2-a565-62fca517e93e" path="/var/lib/kubelet/pods/34ad9193-2191-49f2-a565-62fca517e93e/volumes"
Jul 27 19:56:20.489234 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:20.489213    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46ab011e-dac6-4f34-8c47-7357fb21b24e" path="/var/lib/kubelet/pods/46ab011e-dac6-4f34-8c47-7357fb21b24e/volumes"
Jul 27 19:56:23.386384 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:23.386347    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:23.388703 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:23.386828    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"]
Jul 27 19:56:23.388703 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:23.388531    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6154550c-v85zs"]
Jul 27 19:56:23.567807 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:23.567779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:23.786300 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:23.786238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:24.031784 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:24.031747    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:24.131623 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:24.131527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:24.251344 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:24.251310    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:24.489099 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:24.489066    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f06b73ed-c5f5-4962-83ef-c6101187eb21" path="/var/lib/kubelet/pods/f06b73ed-c5f5-4962-83ef-c6101187eb21/volumes"
Jul 27 19:56:24.550603 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:24.550568    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:25.078002 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:25.077964    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:27.142236 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.142201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:27.250365 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.250336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:27.415980 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.415902    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:27.739063 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.739030    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:27.828366 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.828332    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:56:27.832079 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832054    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"]
Jul 27 19:56:27.832470 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832456    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ab011e-dac6-4f34-8c47-7357fb21b24e" containerName="payload-processing"
Jul 27 19:56:27.832470 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832471    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ab011e-dac6-4f34-8c47-7357fb21b24e" containerName="payload-processing"
Jul 27 19:56:27.832580 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832483    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f06b73ed-c5f5-4962-83ef-c6101187eb21" containerName="revoke-keys"
Jul 27 19:56:27.832580 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832489    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f06b73ed-c5f5-4962-83ef-c6101187eb21" containerName="revoke-keys"
Jul 27 19:56:27.832580 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832503    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ad9193-2191-49f2-a565-62fca517e93e" containerName="payload-pre-processing"
Jul 27 19:56:27.832580 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832509    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ad9193-2191-49f2-a565-62fca517e93e" containerName="payload-pre-processing"
Jul 27 19:56:27.832580 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832562    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="46ab011e-dac6-4f34-8c47-7357fb21b24e" containerName="payload-processing"
Jul 27 19:56:27.832580 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832571    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ad9193-2191-49f2-a565-62fca517e93e" containerName="payload-pre-processing"
Jul 27 19:56:27.832580 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.832577    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f06b73ed-c5f5-4962-83ef-c6101187eb21" containerName="revoke-keys"
Jul 27 19:56:27.835823 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.835803    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:27.838118 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.838093    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-c241bf21\""
Jul 27 19:56:27.838209 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.838097    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-c241bf21-dockercfg-lkjxd\""
Jul 27 19:56:27.844591 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.844571    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"]
Jul 27 19:56:27.869581 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.869557    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"]
Jul 27 19:56:27.873193 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.873171    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:27.885073 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.885051    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"]
Jul 27 19:56:27.947587 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.947559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7hpn\" (UniqueName: \"kubernetes.io/projected/91713bdc-3b83-4d91-8d83-4670848b6f6a-kube-api-access-m7hpn\") pod \"payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z\" (UID: \"91713bdc-3b83-4d91-8d83-4670848b6f6a\") " pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:27.947587 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.947599    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c00612d-0718-4712-8cd2-3569a3142bc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx\" (UID: \"3c00612d-0718-4712-8cd2-3569a3142bc1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:27.947894 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.947649    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmssd\" (UniqueName: \"kubernetes.io/projected/3c00612d-0718-4712-8cd2-3569a3142bc1-kube-api-access-kmssd\") pod \"payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx\" (UID: \"3c00612d-0718-4712-8cd2-3569a3142bc1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:27.947894 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:27.947707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91713bdc-3b83-4d91-8d83-4670848b6f6a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z\" (UID: \"91713bdc-3b83-4d91-8d83-4670848b6f6a\") " pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:28.048754 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.048670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmssd\" (UniqueName: \"kubernetes.io/projected/3c00612d-0718-4712-8cd2-3569a3142bc1-kube-api-access-kmssd\") pod \"payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx\" (UID: \"3c00612d-0718-4712-8cd2-3569a3142bc1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:28.048754 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.048717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91713bdc-3b83-4d91-8d83-4670848b6f6a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z\" (UID: \"91713bdc-3b83-4d91-8d83-4670848b6f6a\") " pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:28.048991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.048769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7hpn\" (UniqueName: \"kubernetes.io/projected/91713bdc-3b83-4d91-8d83-4670848b6f6a-kube-api-access-m7hpn\") pod \"payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z\" (UID: \"91713bdc-3b83-4d91-8d83-4670848b6f6a\") " pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:28.048991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.048796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c00612d-0718-4712-8cd2-3569a3142bc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx\" (UID: \"3c00612d-0718-4712-8cd2-3569a3142bc1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:28.049861 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.049834    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91713bdc-3b83-4d91-8d83-4670848b6f6a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z\" (UID: \"91713bdc-3b83-4d91-8d83-4670848b6f6a\") " pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:28.050006 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.049989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c00612d-0718-4712-8cd2-3569a3142bc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx\" (UID: \"3c00612d-0718-4712-8cd2-3569a3142bc1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:28.056753 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.056727    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7hpn\" (UniqueName: \"kubernetes.io/projected/91713bdc-3b83-4d91-8d83-4670848b6f6a-kube-api-access-m7hpn\") pod \"payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z\" (UID: \"91713bdc-3b83-4d91-8d83-4670848b6f6a\") " pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:28.056872 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.056852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmssd\" (UniqueName: \"kubernetes.io/projected/3c00612d-0718-4712-8cd2-3569a3142bc1-kube-api-access-kmssd\") pod \"payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx\" (UID: \"3c00612d-0718-4712-8cd2-3569a3142bc1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:28.147496 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.147459    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:28.183949 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.183920    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:28.298498 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.298455    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"]
Jul 27 19:56:28.300689 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:56:28.300623    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c00612d_0718_4712_8cd2_3569a3142bc1.slice/crio-86e9d5166fa9042008e9aa7d7f27d2d802d63a7b978a75ca72e054a46b6c35f0 WatchSource:0}: Error finding container 86e9d5166fa9042008e9aa7d7f27d2d802d63a7b978a75ca72e054a46b6c35f0: Status 404 returned error can't find the container with id 86e9d5166fa9042008e9aa7d7f27d2d802d63a7b978a75ca72e054a46b6c35f0
Jul 27 19:56:28.337987 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.337964    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"]
Jul 27 19:56:28.340049 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:56:28.340021    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91713bdc_3b83_4d91_8d83_4670848b6f6a.slice/crio-2ddd8a1984f21ac52dc89d3153234a9c778997e2b3b422f4028c52faf2ee6339 WatchSource:0}: Error finding container 2ddd8a1984f21ac52dc89d3153234a9c778997e2b3b422f4028c52faf2ee6339: Status 404 returned error can't find the container with id 2ddd8a1984f21ac52dc89d3153234a9c778997e2b3b422f4028c52faf2ee6339
Jul 27 19:56:28.994760 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.994725    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z" event={"ID":"91713bdc-3b83-4d91-8d83-4670848b6f6a","Type":"ContainerStarted","Data":"847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244"}
Jul 27 19:56:28.994973 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.994782    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z" event={"ID":"91713bdc-3b83-4d91-8d83-4670848b6f6a","Type":"ContainerStarted","Data":"2ddd8a1984f21ac52dc89d3153234a9c778997e2b3b422f4028c52faf2ee6339"}
Jul 27 19:56:28.994973 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.994841    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:56:28.996370 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.996345    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx" event={"ID":"3c00612d-0718-4712-8cd2-3569a3142bc1","Type":"ContainerStarted","Data":"7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08"}
Jul 27 19:56:28.996484 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.996391    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx" event={"ID":"3c00612d-0718-4712-8cd2-3569a3142bc1","Type":"ContainerStarted","Data":"86e9d5166fa9042008e9aa7d7f27d2d802d63a7b978a75ca72e054a46b6c35f0"}
Jul 27 19:56:28.996484 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:28.996427    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:29.017189 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:29.017122    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z" podStartSLOduration=2.017096742 podStartE2EDuration="2.017096742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:56:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:56:29.013221859 +0000 UTC m=+1841.104573381" watchObservedRunningTime="2026-07-27 19:56:29.017096742 +0000 UTC m=+1841.108448343"
Jul 27 19:56:29.029449 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:29.029399    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx" podStartSLOduration=2.029383071 podStartE2EDuration="2.029383071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:56:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:56:29.028168739 +0000 UTC m=+1841.119520250" watchObservedRunningTime="2026-07-27 19:56:29.029383071 +0000 UTC m=+1841.120734569"
Jul 27 19:56:40.004512 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:40.004475    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:56:40.004977 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:56:40.004556    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:57:02.545734 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.545694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:02.566314 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.566279    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"]
Jul 27 19:57:02.566602 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.566574    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z" podUID="91713bdc-3b83-4d91-8d83-4670848b6f6a" containerName="payload-processing" containerID="cri-o://847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244" gracePeriod=30
Jul 27 19:57:02.570635 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.570319    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"]
Jul 27 19:57:02.570635 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.570530    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx" podUID="3c00612d-0718-4712-8cd2-3569a3142bc1" containerName="payload-pre-processing" containerID="cri-o://7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08" gracePeriod=30
Jul 27 19:57:02.643886 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.643851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:02.843393 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.843372    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:57:02.846353 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.846315    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:57:02.878499 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.878467    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c00612d-0718-4712-8cd2-3569a3142bc1-plugins-config-volume\") pod \"3c00612d-0718-4712-8cd2-3569a3142bc1\" (UID: \"3c00612d-0718-4712-8cd2-3569a3142bc1\") "
Jul 27 19:57:02.878661 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.878524    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7hpn\" (UniqueName: \"kubernetes.io/projected/91713bdc-3b83-4d91-8d83-4670848b6f6a-kube-api-access-m7hpn\") pod \"91713bdc-3b83-4d91-8d83-4670848b6f6a\" (UID: \"91713bdc-3b83-4d91-8d83-4670848b6f6a\") "
Jul 27 19:57:02.878748 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.878644    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91713bdc-3b83-4d91-8d83-4670848b6f6a-plugins-config-volume\") pod \"91713bdc-3b83-4d91-8d83-4670848b6f6a\" (UID: \"91713bdc-3b83-4d91-8d83-4670848b6f6a\") "
Jul 27 19:57:02.878748 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.878717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmssd\" (UniqueName: \"kubernetes.io/projected/3c00612d-0718-4712-8cd2-3569a3142bc1-kube-api-access-kmssd\") pod \"3c00612d-0718-4712-8cd2-3569a3142bc1\" (UID: \"3c00612d-0718-4712-8cd2-3569a3142bc1\") "
Jul 27 19:57:02.879058 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.878971    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c00612d-0718-4712-8cd2-3569a3142bc1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3c00612d-0718-4712-8cd2-3569a3142bc1" (UID: "3c00612d-0718-4712-8cd2-3569a3142bc1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:57:02.879058 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.879000    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91713bdc-3b83-4d91-8d83-4670848b6f6a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "91713bdc-3b83-4d91-8d83-4670848b6f6a" (UID: "91713bdc-3b83-4d91-8d83-4670848b6f6a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:57:02.879232 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.879071    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c00612d-0718-4712-8cd2-3569a3142bc1-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:57:02.880725 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.880664    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91713bdc-3b83-4d91-8d83-4670848b6f6a-kube-api-access-m7hpn" (OuterVolumeSpecName: "kube-api-access-m7hpn") pod "91713bdc-3b83-4d91-8d83-4670848b6f6a" (UID: "91713bdc-3b83-4d91-8d83-4670848b6f6a"). InnerVolumeSpecName "kube-api-access-m7hpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:57:02.880843 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.880766    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c00612d-0718-4712-8cd2-3569a3142bc1-kube-api-access-kmssd" (OuterVolumeSpecName: "kube-api-access-kmssd") pod "3c00612d-0718-4712-8cd2-3569a3142bc1" (UID: "3c00612d-0718-4712-8cd2-3569a3142bc1"). InnerVolumeSpecName "kube-api-access-kmssd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:57:02.979749 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.979718    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7hpn\" (UniqueName: \"kubernetes.io/projected/91713bdc-3b83-4d91-8d83-4670848b6f6a-kube-api-access-m7hpn\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:57:02.979749 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.979744    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91713bdc-3b83-4d91-8d83-4670848b6f6a-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:57:02.979749 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:02.979754    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmssd\" (UniqueName: \"kubernetes.io/projected/3c00612d-0718-4712-8cd2-3569a3142bc1-kube-api-access-kmssd\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:57:03.165266 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.165162    2579 generic.go:356] "Generic (PLEG): container finished" podID="91713bdc-3b83-4d91-8d83-4670848b6f6a" containerID="847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244" exitCode=0
Jul 27 19:57:03.165266 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.165210    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"
Jul 27 19:57:03.165266 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.165237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z" event={"ID":"91713bdc-3b83-4d91-8d83-4670848b6f6a","Type":"ContainerDied","Data":"847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244"}
Jul 27 19:57:03.165535 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.165304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z" event={"ID":"91713bdc-3b83-4d91-8d83-4670848b6f6a","Type":"ContainerDied","Data":"2ddd8a1984f21ac52dc89d3153234a9c778997e2b3b422f4028c52faf2ee6339"}
Jul 27 19:57:03.165535 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.165332    2579 scope.go:117] "RemoveContainer" containerID="847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244"
Jul 27 19:57:03.166672 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.166650    2579 generic.go:356] "Generic (PLEG): container finished" podID="3c00612d-0718-4712-8cd2-3569a3142bc1" containerID="7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08" exitCode=0
Jul 27 19:57:03.166778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.166686    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx" event={"ID":"3c00612d-0718-4712-8cd2-3569a3142bc1","Type":"ContainerDied","Data":"7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08"}
Jul 27 19:57:03.166778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.166698    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"
Jul 27 19:57:03.166778 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.166740    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx" event={"ID":"3c00612d-0718-4712-8cd2-3569a3142bc1","Type":"ContainerDied","Data":"86e9d5166fa9042008e9aa7d7f27d2d802d63a7b978a75ca72e054a46b6c35f0"}
Jul 27 19:57:03.174506 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.174486    2579 scope.go:117] "RemoveContainer" containerID="847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244"
Jul 27 19:57:03.174795 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:57:03.174774    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244\": container with ID starting with 847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244 not found: ID does not exist" containerID="847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244"
Jul 27 19:57:03.174869 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.174802    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244"} err="failed to get container status \"847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244\": rpc error: code = NotFound desc = could not find container \"847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244\": container with ID starting with 847d80b818c554a1b212a9047a4ed19b7ec5d4b512da23df470797e817b6c244 not found: ID does not exist"
Jul 27 19:57:03.174869 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.174819    2579 scope.go:117] "RemoveContainer" containerID="7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08"
Jul 27 19:57:03.182783 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.182765    2579 scope.go:117] "RemoveContainer" containerID="7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08"
Jul 27 19:57:03.183031 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:57:03.183011    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08\": container with ID starting with 7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08 not found: ID does not exist" containerID="7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08"
Jul 27 19:57:03.183094 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.183043    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08"} err="failed to get container status \"7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08\": rpc error: code = NotFound desc = could not find container \"7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08\": container with ID starting with 7e9b6c4f05b4e5ffde8ca93a5a12bc9dcf2aefa8e4295acc75001ca77c99cf08 not found: ID does not exist"
Jul 27 19:57:03.188949 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.188927    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"]
Jul 27 19:57:03.192156 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.192135    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c241bf21-5d648c7fbd-sb9vx"]
Jul 27 19:57:03.202066 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.202047    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"]
Jul 27 19:57:03.206594 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:03.206560    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c241bf21-57778b9465-r6z2z"]
Jul 27 19:57:04.489224 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:04.489192    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c00612d-0718-4712-8cd2-3569a3142bc1" path="/var/lib/kubelet/pods/3c00612d-0718-4712-8cd2-3569a3142bc1/volumes"
Jul 27 19:57:04.489630 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:04.489586    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91713bdc-3b83-4d91-8d83-4670848b6f6a" path="/var/lib/kubelet/pods/91713bdc-3b83-4d91-8d83-4670848b6f6a/volumes"
Jul 27 19:57:07.189226 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:07.189178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:07.340639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:07.340605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:07.551845 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:07.551754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:07.763476 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:07.763441    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:08.079476 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:08.079440    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:08.384177 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:08.384070    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:10.408993 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:10.408914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:10.557529 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:10.557499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:10.751579 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:10.751548    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:10.876870 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:10.876837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:11.285548 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.285521    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:11.353289 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.353254    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"]
Jul 27 19:57:11.353886 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.353857    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91713bdc-3b83-4d91-8d83-4670848b6f6a" containerName="payload-processing"
Jul 27 19:57:11.353886 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.353878    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="91713bdc-3b83-4d91-8d83-4670848b6f6a" containerName="payload-processing"
Jul 27 19:57:11.354022 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.353899    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c00612d-0718-4712-8cd2-3569a3142bc1" containerName="payload-pre-processing"
Jul 27 19:57:11.354022 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.353905    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c00612d-0718-4712-8cd2-3569a3142bc1" containerName="payload-pre-processing"
Jul 27 19:57:11.354022 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.353970    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="91713bdc-3b83-4d91-8d83-4670848b6f6a" containerName="payload-processing"
Jul 27 19:57:11.354022 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.353980    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c00612d-0718-4712-8cd2-3569a3142bc1" containerName="payload-pre-processing"
Jul 27 19:57:11.358977 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.358948    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:11.364005 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.363986    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3ace1558\""
Jul 27 19:57:11.368356 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.368328    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"]
Jul 27 19:57:11.371974 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.371954    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"]
Jul 27 19:57:11.372085 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.372055    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:11.379639 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.379616    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"]
Jul 27 19:57:11.395174 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.395151    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:11.463284 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.463223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bndkn\" (UniqueName: \"kubernetes.io/projected/2e8d3e40-f10e-40d9-997d-d2100e96798d-kube-api-access-bndkn\") pod \"payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts\" (UID: \"2e8d3e40-f10e-40d9-997d-d2100e96798d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:11.463669 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.463337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8d3e40-f10e-40d9-997d-d2100e96798d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts\" (UID: \"2e8d3e40-f10e-40d9-997d-d2100e96798d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:11.463669 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.463405    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc28fb50-7928-41a6-a845-1b187924e840-plugins-config-volume\") pod \"payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn\" (UID: \"dc28fb50-7928-41a6-a845-1b187924e840\") " pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:11.463669 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.463448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv2tn\" (UniqueName: \"kubernetes.io/projected/dc28fb50-7928-41a6-a845-1b187924e840-kube-api-access-pv2tn\") pod \"payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn\" (UID: \"dc28fb50-7928-41a6-a845-1b187924e840\") " pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:11.564356 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.564238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bndkn\" (UniqueName: \"kubernetes.io/projected/2e8d3e40-f10e-40d9-997d-d2100e96798d-kube-api-access-bndkn\") pod \"payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts\" (UID: \"2e8d3e40-f10e-40d9-997d-d2100e96798d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:11.564356 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.564345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8d3e40-f10e-40d9-997d-d2100e96798d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts\" (UID: \"2e8d3e40-f10e-40d9-997d-d2100e96798d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:11.564547 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.564420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc28fb50-7928-41a6-a845-1b187924e840-plugins-config-volume\") pod \"payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn\" (UID: \"dc28fb50-7928-41a6-a845-1b187924e840\") " pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:11.564547 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.564463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv2tn\" (UniqueName: \"kubernetes.io/projected/dc28fb50-7928-41a6-a845-1b187924e840-kube-api-access-pv2tn\") pod \"payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn\" (UID: \"dc28fb50-7928-41a6-a845-1b187924e840\") " pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:11.565044 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.565025    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8d3e40-f10e-40d9-997d-d2100e96798d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts\" (UID: \"2e8d3e40-f10e-40d9-997d-d2100e96798d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:11.565136 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.565118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc28fb50-7928-41a6-a845-1b187924e840-plugins-config-volume\") pod \"payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn\" (UID: \"dc28fb50-7928-41a6-a845-1b187924e840\") " pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:11.572860 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.572837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv2tn\" (UniqueName: \"kubernetes.io/projected/dc28fb50-7928-41a6-a845-1b187924e840-kube-api-access-pv2tn\") pod \"payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn\" (UID: \"dc28fb50-7928-41a6-a845-1b187924e840\") " pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:11.572978 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.572876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bndkn\" (UniqueName: \"kubernetes.io/projected/2e8d3e40-f10e-40d9-997d-d2100e96798d-kube-api-access-bndkn\") pod \"payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts\" (UID: \"2e8d3e40-f10e-40d9-997d-d2100e96798d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:11.671673 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.671636    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:11.683766 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.683732    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:11.842156 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.842133    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"]
Jul 27 19:57:11.843997 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:57:11.843971    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8d3e40_f10e_40d9_997d_d2100e96798d.slice/crio-f6f693b1a8ae367dc138069c1668304b73735a98d423bc83c6f4b486e84c2952 WatchSource:0}: Error finding container f6f693b1a8ae367dc138069c1668304b73735a98d423bc83c6f4b486e84c2952: Status 404 returned error can't find the container with id f6f693b1a8ae367dc138069c1668304b73735a98d423bc83c6f4b486e84c2952
Jul 27 19:57:11.855327 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:11.855307    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"]
Jul 27 19:57:11.857382 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:57:11.857360    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc28fb50_7928_41a6_a845_1b187924e840.slice/crio-5d5684badf74a06fb01b63d089306d8a6cb9c30f086f6c40970574c66e164d07 WatchSource:0}: Error finding container 5d5684badf74a06fb01b63d089306d8a6cb9c30f086f6c40970574c66e164d07: Status 404 returned error can't find the container with id 5d5684badf74a06fb01b63d089306d8a6cb9c30f086f6c40970574c66e164d07
Jul 27 19:57:12.217282 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:12.217224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn" event={"ID":"dc28fb50-7928-41a6-a845-1b187924e840","Type":"ContainerStarted","Data":"6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace"}
Jul 27 19:57:12.217496 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:12.217303    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn" event={"ID":"dc28fb50-7928-41a6-a845-1b187924e840","Type":"ContainerStarted","Data":"5d5684badf74a06fb01b63d089306d8a6cb9c30f086f6c40970574c66e164d07"}
Jul 27 19:57:12.217496 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:12.217345    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:12.218821 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:12.218793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts" event={"ID":"2e8d3e40-f10e-40d9-997d-d2100e96798d","Type":"ContainerStarted","Data":"d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a"}
Jul 27 19:57:12.218965 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:12.218836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts" event={"ID":"2e8d3e40-f10e-40d9-997d-d2100e96798d","Type":"ContainerStarted","Data":"f6f693b1a8ae367dc138069c1668304b73735a98d423bc83c6f4b486e84c2952"}
Jul 27 19:57:12.218965 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:12.218890    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:12.236150 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:12.236086    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn" podStartSLOduration=1.236066179 podStartE2EDuration="1.236066179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:57:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:57:12.233927947 +0000 UTC m=+1884.325279446" watchObservedRunningTime="2026-07-27 19:57:12.236066179 +0000 UTC m=+1884.327417692"
Jul 27 19:57:12.254696 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:12.254633    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts" podStartSLOduration=1.254613203 podStartE2EDuration="1.254613203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:57:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:57:12.252141381 +0000 UTC m=+1884.343492881" watchObservedRunningTime="2026-07-27 19:57:12.254613203 +0000 UTC m=+1884.345964703"
Jul 27 19:57:23.226084 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:23.226055    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:23.226658 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:23.226276    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:46.616453 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.616419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:46.637377 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.637340    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"]
Jul 27 19:57:46.637626 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.637598    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn" podUID="dc28fb50-7928-41a6-a845-1b187924e840" containerName="payload-processing" containerID="cri-o://6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace" gracePeriod=30
Jul 27 19:57:46.642232 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.642073    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"]
Jul 27 19:57:46.642417 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.642342    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts" podUID="2e8d3e40-f10e-40d9-997d-d2100e96798d" containerName="payload-pre-processing" containerID="cri-o://d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a" gracePeriod=30
Jul 27 19:57:46.719170 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.719138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:46.916748 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.916721    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:46.919689 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.919668    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:46.998982 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.998945    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bndkn\" (UniqueName: \"kubernetes.io/projected/2e8d3e40-f10e-40d9-997d-d2100e96798d-kube-api-access-bndkn\") pod \"2e8d3e40-f10e-40d9-997d-d2100e96798d\" (UID: \"2e8d3e40-f10e-40d9-997d-d2100e96798d\") "
Jul 27 19:57:46.999185 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.999001    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc28fb50-7928-41a6-a845-1b187924e840-plugins-config-volume\") pod \"dc28fb50-7928-41a6-a845-1b187924e840\" (UID: \"dc28fb50-7928-41a6-a845-1b187924e840\") "
Jul 27 19:57:46.999185 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.999138    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8d3e40-f10e-40d9-997d-d2100e96798d-plugins-config-volume\") pod \"2e8d3e40-f10e-40d9-997d-d2100e96798d\" (UID: \"2e8d3e40-f10e-40d9-997d-d2100e96798d\") "
Jul 27 19:57:46.999356 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.999296    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv2tn\" (UniqueName: \"kubernetes.io/projected/dc28fb50-7928-41a6-a845-1b187924e840-kube-api-access-pv2tn\") pod \"dc28fb50-7928-41a6-a845-1b187924e840\" (UID: \"dc28fb50-7928-41a6-a845-1b187924e840\") "
Jul 27 19:57:46.999539 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.999511    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e8d3e40-f10e-40d9-997d-d2100e96798d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e8d3e40-f10e-40d9-997d-d2100e96798d" (UID: "2e8d3e40-f10e-40d9-997d-d2100e96798d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:57:46.999706 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.999479    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc28fb50-7928-41a6-a845-1b187924e840-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc28fb50-7928-41a6-a845-1b187924e840" (UID: "dc28fb50-7928-41a6-a845-1b187924e840"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:57:46.999706 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.999663    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc28fb50-7928-41a6-a845-1b187924e840-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:57:46.999706 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:46.999683    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e8d3e40-f10e-40d9-997d-d2100e96798d-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:57:47.001262 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.001219    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e8d3e40-f10e-40d9-997d-d2100e96798d-kube-api-access-bndkn" (OuterVolumeSpecName: "kube-api-access-bndkn") pod "2e8d3e40-f10e-40d9-997d-d2100e96798d" (UID: "2e8d3e40-f10e-40d9-997d-d2100e96798d"). InnerVolumeSpecName "kube-api-access-bndkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:57:47.001378 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.001359    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc28fb50-7928-41a6-a845-1b187924e840-kube-api-access-pv2tn" (OuterVolumeSpecName: "kube-api-access-pv2tn") pod "dc28fb50-7928-41a6-a845-1b187924e840" (UID: "dc28fb50-7928-41a6-a845-1b187924e840"). InnerVolumeSpecName "kube-api-access-pv2tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:57:47.100790 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.100756    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv2tn\" (UniqueName: \"kubernetes.io/projected/dc28fb50-7928-41a6-a845-1b187924e840-kube-api-access-pv2tn\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:57:47.100790 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.100784    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bndkn\" (UniqueName: \"kubernetes.io/projected/2e8d3e40-f10e-40d9-997d-d2100e96798d-kube-api-access-bndkn\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:57:47.392730 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.392697    2579 generic.go:356] "Generic (PLEG): container finished" podID="dc28fb50-7928-41a6-a845-1b187924e840" containerID="6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace" exitCode=0
Jul 27 19:57:47.392922 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.392755    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"
Jul 27 19:57:47.392922 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.392781    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn" event={"ID":"dc28fb50-7928-41a6-a845-1b187924e840","Type":"ContainerDied","Data":"6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace"}
Jul 27 19:57:47.392922 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.392851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn" event={"ID":"dc28fb50-7928-41a6-a845-1b187924e840","Type":"ContainerDied","Data":"5d5684badf74a06fb01b63d089306d8a6cb9c30f086f6c40970574c66e164d07"}
Jul 27 19:57:47.392922 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.392886    2579 scope.go:117] "RemoveContainer" containerID="6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace"
Jul 27 19:57:47.394133 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.394106    2579 generic.go:356] "Generic (PLEG): container finished" podID="2e8d3e40-f10e-40d9-997d-d2100e96798d" containerID="d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a" exitCode=0
Jul 27 19:57:47.394274 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.394135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts" event={"ID":"2e8d3e40-f10e-40d9-997d-d2100e96798d","Type":"ContainerDied","Data":"d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a"}
Jul 27 19:57:47.394274 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.394170    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"
Jul 27 19:57:47.394274 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.394194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts" event={"ID":"2e8d3e40-f10e-40d9-997d-d2100e96798d","Type":"ContainerDied","Data":"f6f693b1a8ae367dc138069c1668304b73735a98d423bc83c6f4b486e84c2952"}
Jul 27 19:57:47.402891 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.402872    2579 scope.go:117] "RemoveContainer" containerID="6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace"
Jul 27 19:57:47.403179 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:57:47.403153    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace\": container with ID starting with 6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace not found: ID does not exist" containerID="6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace"
Jul 27 19:57:47.403330 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.403188    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace"} err="failed to get container status \"6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace\": rpc error: code = NotFound desc = could not find container \"6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace\": container with ID starting with 6df15e626b37513b7d3a5d94fac51875df692eb25e202853f7ffb54b67173ace not found: ID does not exist"
Jul 27 19:57:47.403330 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.403210    2579 scope.go:117] "RemoveContainer" containerID="d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a"
Jul 27 19:57:47.411431 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.411415    2579 scope.go:117] "RemoveContainer" containerID="d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a"
Jul 27 19:57:47.411696 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:57:47.411666    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a\": container with ID starting with d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a not found: ID does not exist" containerID="d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a"
Jul 27 19:57:47.411758 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.411692    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a"} err="failed to get container status \"d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a\": rpc error: code = NotFound desc = could not find container \"d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a\": container with ID starting with d7b8780e6c0dd4d3b87a0bd3eb05830d315ee462fe6f05d17b03e793dda0c82a not found: ID does not exist"
Jul 27 19:57:47.418715 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.418694    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"]
Jul 27 19:57:47.424218 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.424195    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3ace1558-547475bcbd-7hxts"]
Jul 27 19:57:47.434046 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.434024    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"]
Jul 27 19:57:47.436898 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:47.436877    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3ace1558-7c8945dfd6-26tjn"]
Jul 27 19:57:48.493507 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:48.493466    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e8d3e40-f10e-40d9-997d-d2100e96798d" path="/var/lib/kubelet/pods/2e8d3e40-f10e-40d9-997d-d2100e96798d/volumes"
Jul 27 19:57:48.494057 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:48.494033    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc28fb50-7928-41a6-a845-1b187924e840" path="/var/lib/kubelet/pods/dc28fb50-7928-41a6-a845-1b187924e840/volumes"
Jul 27 19:57:51.168159 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:51.168074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:52.345909 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:52.345877    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:52.555499 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:52.555471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:52.768380 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:52.768350    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:53.057041 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:53.056967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:53.572610 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:53.572549    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:55.513013 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:55.512977    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:55.586487 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:55.586451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:55.667833 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:55.667803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:55.768534 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:55.768450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:56.195292 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.195263    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:56.283353 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.283317    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"]
Jul 27 19:57:56.283823 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.283794    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e8d3e40-f10e-40d9-997d-d2100e96798d" containerName="payload-pre-processing"
Jul 27 19:57:56.283823 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.283815    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e8d3e40-f10e-40d9-997d-d2100e96798d" containerName="payload-pre-processing"
Jul 27 19:57:56.283991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.283833    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc28fb50-7928-41a6-a845-1b187924e840" containerName="payload-processing"
Jul 27 19:57:56.283991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.283842    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc28fb50-7928-41a6-a845-1b187924e840" containerName="payload-processing"
Jul 27 19:57:56.283991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.283930    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc28fb50-7928-41a6-a845-1b187924e840" containerName="payload-processing"
Jul 27 19:57:56.283991 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.283948    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e8d3e40-f10e-40d9-997d-d2100e96798d" containerName="payload-pre-processing"
Jul 27 19:57:56.286892 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.286872    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:56.289354 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.289333    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-6ac18d4b\""
Jul 27 19:57:56.296682 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.296661    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"]
Jul 27 19:57:56.300227 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.300207    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:57:56.318228 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.318195    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"]
Jul 27 19:57:56.322505 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.322485    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:56.324959 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.324927    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-6ac18d4b-dockercfg-h7xfl\""
Jul 27 19:57:56.328943 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.328922    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"]
Jul 27 19:57:56.390115 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.390079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-plugins-config-volume\") pod \"payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg\" (UID: \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\") " pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:56.390345 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.390124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lrsm\" (UniqueName: \"kubernetes.io/projected/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-kube-api-access-9lrsm\") pod \"payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg\" (UID: \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\") " pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:56.390345 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.390312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9xz\" (UniqueName: \"kubernetes.io/projected/0def6054-63ff-4867-9f69-d7bad4aa1398-kube-api-access-nw9xz\") pod \"payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk\" (UID: \"0def6054-63ff-4867-9f69-d7bad4aa1398\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:56.390474 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.390373    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0def6054-63ff-4867-9f69-d7bad4aa1398-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk\" (UID: \"0def6054-63ff-4867-9f69-d7bad4aa1398\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:56.491575 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.491500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9xz\" (UniqueName: \"kubernetes.io/projected/0def6054-63ff-4867-9f69-d7bad4aa1398-kube-api-access-nw9xz\") pod \"payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk\" (UID: \"0def6054-63ff-4867-9f69-d7bad4aa1398\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:56.491575 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.491549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0def6054-63ff-4867-9f69-d7bad4aa1398-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk\" (UID: \"0def6054-63ff-4867-9f69-d7bad4aa1398\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:56.491815 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.491580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-plugins-config-volume\") pod \"payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg\" (UID: \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\") " pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:56.491815 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.491605    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lrsm\" (UniqueName: \"kubernetes.io/projected/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-kube-api-access-9lrsm\") pod \"payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg\" (UID: \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\") " pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:56.492362 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.492340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0def6054-63ff-4867-9f69-d7bad4aa1398-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk\" (UID: \"0def6054-63ff-4867-9f69-d7bad4aa1398\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:56.492462 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.492340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-plugins-config-volume\") pod \"payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg\" (UID: \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\") " pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:56.500532 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.500509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9xz\" (UniqueName: \"kubernetes.io/projected/0def6054-63ff-4867-9f69-d7bad4aa1398-kube-api-access-nw9xz\") pod \"payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk\" (UID: \"0def6054-63ff-4867-9f69-d7bad4aa1398\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:56.500655 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.500634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lrsm\" (UniqueName: \"kubernetes.io/projected/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-kube-api-access-9lrsm\") pod \"payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg\" (UID: \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\") " pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:56.598396 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.598357    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:56.634742 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.634707    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:56.754966 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.754871    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"]
Jul 27 19:57:56.758926 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:57:56.758893    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0def6054_63ff_4867_9f69_d7bad4aa1398.slice/crio-1ace71842f942340341b1443da8b21fd88b74865c022afa0adc4e146ae15b990 WatchSource:0}: Error finding container 1ace71842f942340341b1443da8b21fd88b74865c022afa0adc4e146ae15b990: Status 404 returned error can't find the container with id 1ace71842f942340341b1443da8b21fd88b74865c022afa0adc4e146ae15b990
Jul 27 19:57:56.795270 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:56.795225    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"]
Jul 27 19:57:56.798041 ip-10-0-131-59 kubenswrapper[2579]: W0727 19:57:56.798016    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf06dc7d5_5f13_47ec_b45b_4f3f84d36c08.slice/crio-e5436a44f332614b2c2d8476c41f38bf85763d36dbbd25b32763ba2dc98845bd WatchSource:0}: Error finding container e5436a44f332614b2c2d8476c41f38bf85763d36dbbd25b32763ba2dc98845bd: Status 404 returned error can't find the container with id e5436a44f332614b2c2d8476c41f38bf85763d36dbbd25b32763ba2dc98845bd
Jul 27 19:57:57.446418 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:57.446378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg" event={"ID":"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08","Type":"ContainerStarted","Data":"eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359"}
Jul 27 19:57:57.446640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:57.446447    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg" event={"ID":"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08","Type":"ContainerStarted","Data":"e5436a44f332614b2c2d8476c41f38bf85763d36dbbd25b32763ba2dc98845bd"}
Jul 27 19:57:57.446640 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:57.446480    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:57:57.448129 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:57.448102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk" event={"ID":"0def6054-63ff-4867-9f69-d7bad4aa1398","Type":"ContainerStarted","Data":"1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca"}
Jul 27 19:57:57.448271 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:57.448151    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk" event={"ID":"0def6054-63ff-4867-9f69-d7bad4aa1398","Type":"ContainerStarted","Data":"1ace71842f942340341b1443da8b21fd88b74865c022afa0adc4e146ae15b990"}
Jul 27 19:57:57.448271 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:57.448259    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:57:57.463702 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:57.463660    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg" podStartSLOduration=1.463644623 podStartE2EDuration="1.463644623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:57:57.461790536 +0000 UTC m=+1929.553142057" watchObservedRunningTime="2026-07-27 19:57:57.463644623 +0000 UTC m=+1929.554996172"
Jul 27 19:57:57.479730 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:57:57.479684    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk" podStartSLOduration=1.4796689619999999 podStartE2EDuration="1.479668962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:57:57.4774755 +0000 UTC m=+1929.568826998" watchObservedRunningTime="2026-07-27 19:57:57.479668962 +0000 UTC m=+1929.571020460"
Jul 27 19:58:08.455911 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:08.455878    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:58:08.456379 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:08.455951    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:58:30.789261 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:30.789196    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:30.802732 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:30.802700    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"]
Jul 27 19:58:30.803016 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:30.802984    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg" podUID="f06dc7d5-5f13-47ec-b45b-4f3f84d36c08" containerName="payload-processing" containerID="cri-o://eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359" gracePeriod=30
Jul 27 19:58:30.811339 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:30.811313    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"]
Jul 27 19:58:30.811821 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:30.811601    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk" podUID="0def6054-63ff-4867-9f69-d7bad4aa1398" containerName="payload-pre-processing" containerID="cri-o://1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca" gracePeriod=30
Jul 27 19:58:30.879001 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:30.878957    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:31.102613 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.102584    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:58:31.105695 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.105672    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:58:31.225034 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.225003    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-plugins-config-volume\") pod \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\" (UID: \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\") "
Jul 27 19:58:31.225266 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.225058    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lrsm\" (UniqueName: \"kubernetes.io/projected/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-kube-api-access-9lrsm\") pod \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\" (UID: \"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08\") "
Jul 27 19:58:31.225266 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.225107    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw9xz\" (UniqueName: \"kubernetes.io/projected/0def6054-63ff-4867-9f69-d7bad4aa1398-kube-api-access-nw9xz\") pod \"0def6054-63ff-4867-9f69-d7bad4aa1398\" (UID: \"0def6054-63ff-4867-9f69-d7bad4aa1398\") "
Jul 27 19:58:31.225266 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.225140    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0def6054-63ff-4867-9f69-d7bad4aa1398-plugins-config-volume\") pod \"0def6054-63ff-4867-9f69-d7bad4aa1398\" (UID: \"0def6054-63ff-4867-9f69-d7bad4aa1398\") "
Jul 27 19:58:31.225539 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.225513    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f06dc7d5-5f13-47ec-b45b-4f3f84d36c08" (UID: "f06dc7d5-5f13-47ec-b45b-4f3f84d36c08"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:58:31.225627 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.225594    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0def6054-63ff-4867-9f69-d7bad4aa1398-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0def6054-63ff-4867-9f69-d7bad4aa1398" (UID: "0def6054-63ff-4867-9f69-d7bad4aa1398"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:58:31.227431 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.227401    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-kube-api-access-9lrsm" (OuterVolumeSpecName: "kube-api-access-9lrsm") pod "f06dc7d5-5f13-47ec-b45b-4f3f84d36c08" (UID: "f06dc7d5-5f13-47ec-b45b-4f3f84d36c08"). InnerVolumeSpecName "kube-api-access-9lrsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:58:31.227537 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.227473    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0def6054-63ff-4867-9f69-d7bad4aa1398-kube-api-access-nw9xz" (OuterVolumeSpecName: "kube-api-access-nw9xz") pod "0def6054-63ff-4867-9f69-d7bad4aa1398" (UID: "0def6054-63ff-4867-9f69-d7bad4aa1398"). InnerVolumeSpecName "kube-api-access-nw9xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:58:31.326446 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.326357    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lrsm\" (UniqueName: \"kubernetes.io/projected/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-kube-api-access-9lrsm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:58:31.326446 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.326389    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw9xz\" (UniqueName: \"kubernetes.io/projected/0def6054-63ff-4867-9f69-d7bad4aa1398-kube-api-access-nw9xz\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:58:31.326446 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.326399    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0def6054-63ff-4867-9f69-d7bad4aa1398-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:58:31.326446 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.326409    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 19:58:31.627145 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.627055    2579 generic.go:356] "Generic (PLEG): container finished" podID="f06dc7d5-5f13-47ec-b45b-4f3f84d36c08" containerID="eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359" exitCode=0
Jul 27 19:58:31.627145 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.627120    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"
Jul 27 19:58:31.627145 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.627120    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg" event={"ID":"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08","Type":"ContainerDied","Data":"eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359"}
Jul 27 19:58:31.627432 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.627199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg" event={"ID":"f06dc7d5-5f13-47ec-b45b-4f3f84d36c08","Type":"ContainerDied","Data":"e5436a44f332614b2c2d8476c41f38bf85763d36dbbd25b32763ba2dc98845bd"}
Jul 27 19:58:31.627432 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.627229    2579 scope.go:117] "RemoveContainer" containerID="eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359"
Jul 27 19:58:31.628730 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.628706    2579 generic.go:356] "Generic (PLEG): container finished" podID="0def6054-63ff-4867-9f69-d7bad4aa1398" containerID="1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca" exitCode=0
Jul 27 19:58:31.628818 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.628753    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"
Jul 27 19:58:31.628818 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.628756    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk" event={"ID":"0def6054-63ff-4867-9f69-d7bad4aa1398","Type":"ContainerDied","Data":"1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca"}
Jul 27 19:58:31.628818 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.628800    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk" event={"ID":"0def6054-63ff-4867-9f69-d7bad4aa1398","Type":"ContainerDied","Data":"1ace71842f942340341b1443da8b21fd88b74865c022afa0adc4e146ae15b990"}
Jul 27 19:58:31.636863 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.636845    2579 scope.go:117] "RemoveContainer" containerID="eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359"
Jul 27 19:58:31.637232 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:58:31.637205    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359\": container with ID starting with eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359 not found: ID does not exist" containerID="eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359"
Jul 27 19:58:31.637345 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.637259    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359"} err="failed to get container status \"eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359\": rpc error: code = NotFound desc = could not find container \"eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359\": container with ID starting with eb083be015f0cb7ea007322015adb10171992d99648aecd0174091e6c7ab1359 not found: ID does not exist"
Jul 27 19:58:31.637345 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.637285    2579 scope.go:117] "RemoveContainer" containerID="1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca"
Jul 27 19:58:31.645147 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.645132    2579 scope.go:117] "RemoveContainer" containerID="1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca"
Jul 27 19:58:31.645423 ip-10-0-131-59 kubenswrapper[2579]: E0727 19:58:31.645405    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca\": container with ID starting with 1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca not found: ID does not exist" containerID="1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca"
Jul 27 19:58:31.645491 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.645429    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca"} err="failed to get container status \"1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca\": rpc error: code = NotFound desc = could not find container \"1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca\": container with ID starting with 1894a81c39d19cc94663b674e8deb30720df3222c66af35c78a42095bbfd22ca not found: ID does not exist"
Jul 27 19:58:31.652048 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.652028    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"]
Jul 27 19:58:31.655376 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.655356    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6ac18d4b-7559cc4644-lv2rk"]
Jul 27 19:58:31.664649 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.664628    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"]
Jul 27 19:58:31.667976 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:31.667956    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6ac18d4b-7d78889f8-llvrg"]
Jul 27 19:58:32.488845 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:32.488813    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0def6054-63ff-4867-9f69-d7bad4aa1398" path="/var/lib/kubelet/pods/0def6054-63ff-4867-9f69-d7bad4aa1398/volumes"
Jul 27 19:58:32.489222 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:32.489177    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f06dc7d5-5f13-47ec-b45b-4f3f84d36c08" path="/var/lib/kubelet/pods/f06dc7d5-5f13-47ec-b45b-4f3f84d36c08/volumes"
Jul 27 19:58:35.474255 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:35.474219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:35.624740 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:35.624706    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:35.850751 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:35.850668    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:36.304153 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:36.304118    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:36.404556 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:36.404523    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:36.505026 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:36.504996    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:37.884015 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:37.883980    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:44.192557 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:44.192472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:44.319897 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:44.319860    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:44.411626 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:44.411590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:44.511212 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:44.511097    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:58:44.871164 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:58:44.871063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:10.814967 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:10.814931    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:17.400994 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:17.400963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:17.502072 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:17.502032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:17.601437 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:17.601401    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:31.248415 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:31.248375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:33.461380 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:33.461340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:39.511350 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:39.511306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:39.614222 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:39.614187    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:39.716554 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:39.716521    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:50.190107 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:50.190076    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:50.974397 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:50.974360    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:57.716582 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:57.716546    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:57.830056 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:57.830024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:57.932825 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:57.932795    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:58.029304 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:58.029207    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:58.758697 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:58.758664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:58.865871 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:58.865838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 19:59:58.966436 ip-10-0-131-59 kubenswrapper[2579]: I0727 19:59:58.966400    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:00.142119 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.142089    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm"]
Jul 27 20:00:00.142562 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.142522    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0def6054-63ff-4867-9f69-d7bad4aa1398" containerName="payload-pre-processing"
Jul 27 20:00:00.142562 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.142538    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0def6054-63ff-4867-9f69-d7bad4aa1398" containerName="payload-pre-processing"
Jul 27 20:00:00.142562 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.142546    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f06dc7d5-5f13-47ec-b45b-4f3f84d36c08" containerName="payload-processing"
Jul 27 20:00:00.142562 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.142552    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f06dc7d5-5f13-47ec-b45b-4f3f84d36c08" containerName="payload-processing"
Jul 27 20:00:00.142725 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.142618    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0def6054-63ff-4867-9f69-d7bad4aa1398" containerName="payload-pre-processing"
Jul 27 20:00:00.142725 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.142626    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f06dc7d5-5f13-47ec-b45b-4f3f84d36c08" containerName="payload-processing"
Jul 27 20:00:00.145672 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.145652    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm"
Jul 27 20:00:00.148337 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.148314    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 20:00:00.152990 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.152969    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm"]
Jul 27 20:00:00.157106 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.157086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm"
Jul 27 20:00:00.290439 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.289877    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm"]
Jul 27 20:00:00.293162 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:00:00.293132    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390e5a62_bbea_444b_8f35_d818ef99b754.slice/crio-6390ff9480d77fb5740262c7a3c3108413a1fc814c75d5ee70ae28dfbf9aaefc WatchSource:0}: Error finding container 6390ff9480d77fb5740262c7a3c3108413a1fc814c75d5ee70ae28dfbf9aaefc: Status 404 returned error can't find the container with id 6390ff9480d77fb5740262c7a3c3108413a1fc814c75d5ee70ae28dfbf9aaefc
Jul 27 20:00:00.661649 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:00.661620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:01.062862 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:01.062828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm" event={"ID":"390e5a62-bbea-444b-8f35-d818ef99b754","Type":"ContainerStarted","Data":"35195ec63dc661417f4f9395e9bfadf5d87dc433b9fb8bf7c18a9e263476d5c5"}
Jul 27 20:00:01.063098 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:01.062890    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm" event={"ID":"390e5a62-bbea-444b-8f35-d818ef99b754","Type":"ContainerStarted","Data":"6390ff9480d77fb5740262c7a3c3108413a1fc814c75d5ee70ae28dfbf9aaefc"}
Jul 27 20:00:01.079679 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:01.079625    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm" podStartSLOduration=1.079608367 podStartE2EDuration="1.079608367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:00:01.077927555 +0000 UTC m=+2053.169279077" watchObservedRunningTime="2026-07-27 20:00:01.079608367 +0000 UTC m=+2053.170959865"
Jul 27 20:00:02.068885 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:02.068848    2579 generic.go:356] "Generic (PLEG): container finished" podID="390e5a62-bbea-444b-8f35-d818ef99b754" containerID="35195ec63dc661417f4f9395e9bfadf5d87dc433b9fb8bf7c18a9e263476d5c5" exitCode=0
Jul 27 20:00:02.069308 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:02.068935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm" event={"ID":"390e5a62-bbea-444b-8f35-d818ef99b754","Type":"ContainerDied","Data":"35195ec63dc661417f4f9395e9bfadf5d87dc433b9fb8bf7c18a9e263476d5c5"}
Jul 27 20:00:03.203375 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:03.203353    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm"
Jul 27 20:00:04.080869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:04.080834    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm" event={"ID":"390e5a62-bbea-444b-8f35-d818ef99b754","Type":"ContainerDied","Data":"6390ff9480d77fb5740262c7a3c3108413a1fc814c75d5ee70ae28dfbf9aaefc"}
Jul 27 20:00:04.081049 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:04.080887    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6390ff9480d77fb5740262c7a3c3108413a1fc814c75d5ee70ae28dfbf9aaefc"
Jul 27 20:00:04.081049 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:04.080842    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753040-722gm"
Jul 27 20:00:06.380087 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:06.380048    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:06.462721 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:06.462689    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:07.070982 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:07.070945    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:13.427624 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:13.427589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:13.527424 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:13.527384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:13.689710 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:13.689682    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:14.001929 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:14.001846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:14.189306 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:14.189273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:16.894083 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:16.894046    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:16.993902 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:16.993872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:17.132494 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.132461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:17.585274 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.585220    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:17.679106 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.679076    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:17.682713 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.682688    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"]
Jul 27 20:00:17.683066 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.683055    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390e5a62-bbea-444b-8f35-d818ef99b754" containerName="cleanup"
Jul 27 20:00:17.683113 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.683068    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="390e5a62-bbea-444b-8f35-d818ef99b754" containerName="cleanup"
Jul 27 20:00:17.683148 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.683142    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="390e5a62-bbea-444b-8f35-d818ef99b754" containerName="cleanup"
Jul 27 20:00:17.686409 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.686380    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:17.688712 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.688691    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-61128fd0\""
Jul 27 20:00:17.688833 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.688818    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-61128fd0-dockercfg-cbwkm\""
Jul 27 20:00:17.696008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.695980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d478g\" (UniqueName: \"kubernetes.io/projected/00b65138-c029-44eb-aa63-08c822dc0cbc-kube-api-access-d478g\") pod \"payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc\" (UID: \"00b65138-c029-44eb-aa63-08c822dc0cbc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:17.696136 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.696078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b65138-c029-44eb-aa63-08c822dc0cbc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc\" (UID: \"00b65138-c029-44eb-aa63-08c822dc0cbc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:17.697489 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.697469    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"]
Jul 27 20:00:17.721698 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.721668    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"]
Jul 27 20:00:17.725376 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.725355    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:17.734926 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.734899    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"]
Jul 27 20:00:17.748256 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.748210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:17.797071 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.797033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d478g\" (UniqueName: \"kubernetes.io/projected/00b65138-c029-44eb-aa63-08c822dc0cbc-kube-api-access-d478g\") pod \"payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc\" (UID: \"00b65138-c029-44eb-aa63-08c822dc0cbc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:17.797215 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.797074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b65138-c029-44eb-aa63-08c822dc0cbc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc\" (UID: \"00b65138-c029-44eb-aa63-08c822dc0cbc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:17.797801 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.797783    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b65138-c029-44eb-aa63-08c822dc0cbc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc\" (UID: \"00b65138-c029-44eb-aa63-08c822dc0cbc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:17.804713 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.804685    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d478g\" (UniqueName: \"kubernetes.io/projected/00b65138-c029-44eb-aa63-08c822dc0cbc-kube-api-access-d478g\") pod \"payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc\" (UID: \"00b65138-c029-44eb-aa63-08c822dc0cbc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:17.898548 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.898456    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f212310f-484e-4abf-a368-97e7193cc841-plugins-config-volume\") pod \"payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6\" (UID: \"f212310f-484e-4abf-a368-97e7193cc841\") " pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:17.898905 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.898570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crpzc\" (UniqueName: \"kubernetes.io/projected/f212310f-484e-4abf-a368-97e7193cc841-kube-api-access-crpzc\") pod \"payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6\" (UID: \"f212310f-484e-4abf-a368-97e7193cc841\") " pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:17.997977 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.997937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:17.999219 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.999194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f212310f-484e-4abf-a368-97e7193cc841-plugins-config-volume\") pod \"payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6\" (UID: \"f212310f-484e-4abf-a368-97e7193cc841\") " pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:17.999351 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.999286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crpzc\" (UniqueName: \"kubernetes.io/projected/f212310f-484e-4abf-a368-97e7193cc841-kube-api-access-crpzc\") pod \"payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6\" (UID: \"f212310f-484e-4abf-a368-97e7193cc841\") " pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:17.999877 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:17.999857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f212310f-484e-4abf-a368-97e7193cc841-plugins-config-volume\") pod \"payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6\" (UID: \"f212310f-484e-4abf-a368-97e7193cc841\") " pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:18.006752 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:18.006726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crpzc\" (UniqueName: \"kubernetes.io/projected/f212310f-484e-4abf-a368-97e7193cc841-kube-api-access-crpzc\") pod \"payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6\" (UID: \"f212310f-484e-4abf-a368-97e7193cc841\") " pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:18.037425 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:18.037383    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:18.146886 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:18.146852    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"]
Jul 27 20:00:18.150179 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:00:18.150104    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b65138_c029_44eb_aa63_08c822dc0cbc.slice/crio-71d12298e9d9f0f3a4e36a955cf9bfe5742e517950de2ef242574f6513e75efc WatchSource:0}: Error finding container 71d12298e9d9f0f3a4e36a955cf9bfe5742e517950de2ef242574f6513e75efc: Status 404 returned error can't find the container with id 71d12298e9d9f0f3a4e36a955cf9bfe5742e517950de2ef242574f6513e75efc
Jul 27 20:00:18.193367 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:18.193332    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"]
Jul 27 20:00:18.206346 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:00:18.206315    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf212310f_484e_4abf_a368_97e7193cc841.slice/crio-9adf93f4288d3d9d8b74496ac724ec410537571238a93e754d328307c1efa246 WatchSource:0}: Error finding container 9adf93f4288d3d9d8b74496ac724ec410537571238a93e754d328307c1efa246: Status 404 returned error can't find the container with id 9adf93f4288d3d9d8b74496ac724ec410537571238a93e754d328307c1efa246
Jul 27 20:00:19.159255 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:19.159199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6" event={"ID":"f212310f-484e-4abf-a368-97e7193cc841","Type":"ContainerStarted","Data":"d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f"}
Jul 27 20:00:19.159733 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:19.159397    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:19.159733 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:19.159469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6" event={"ID":"f212310f-484e-4abf-a368-97e7193cc841","Type":"ContainerStarted","Data":"9adf93f4288d3d9d8b74496ac724ec410537571238a93e754d328307c1efa246"}
Jul 27 20:00:19.160928 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:19.160905    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc" event={"ID":"00b65138-c029-44eb-aa63-08c822dc0cbc","Type":"ContainerStarted","Data":"8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31"}
Jul 27 20:00:19.161071 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:19.160948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc" event={"ID":"00b65138-c029-44eb-aa63-08c822dc0cbc","Type":"ContainerStarted","Data":"71d12298e9d9f0f3a4e36a955cf9bfe5742e517950de2ef242574f6513e75efc"}
Jul 27 20:00:19.161071 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:19.160979    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:19.176448 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:19.176396    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6" podStartSLOduration=2.17637628 podStartE2EDuration="2.17637628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:00:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:00:19.175392501 +0000 UTC m=+2071.266744000" watchObservedRunningTime="2026-07-27 20:00:19.17637628 +0000 UTC m=+2071.267727780"
Jul 27 20:00:19.190916 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:19.190859    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc" podStartSLOduration=2.190844012 podStartE2EDuration="2.190844012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:00:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:00:19.189650788 +0000 UTC m=+2071.281002303" watchObservedRunningTime="2026-07-27 20:00:19.190844012 +0000 UTC m=+2071.282195510"
Jul 27 20:00:21.441967 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:21.441935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:22.127618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:22.127572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:22.226651 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:22.226617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:22.555966 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:22.555935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:28.518050 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:28.518009    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:28.876695 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:28.876616    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:30.174132 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:30.174105    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:30.174566 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:30.174172    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:39.384223 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.384190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:39.475757 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.475718    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"]
Jul 27 20:00:39.476022 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.475995    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6" podUID="f212310f-484e-4abf-a368-97e7193cc841" containerName="payload-processing" containerID="cri-o://d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f" gracePeriod=30
Jul 27 20:00:39.482512 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.482472    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"]
Jul 27 20:00:39.482712 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.482685    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc" podUID="00b65138-c029-44eb-aa63-08c822dc0cbc" containerName="payload-pre-processing" containerID="cri-o://8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31" gracePeriod=30
Jul 27 20:00:39.748967 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.748934    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:39.753128 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.753109    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:39.785285 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.785236    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:39.806569 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.806542    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f212310f-484e-4abf-a368-97e7193cc841-plugins-config-volume\") pod \"f212310f-484e-4abf-a368-97e7193cc841\" (UID: \"f212310f-484e-4abf-a368-97e7193cc841\") "
Jul 27 20:00:39.806743 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.806705    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crpzc\" (UniqueName: \"kubernetes.io/projected/f212310f-484e-4abf-a368-97e7193cc841-kube-api-access-crpzc\") pod \"f212310f-484e-4abf-a368-97e7193cc841\" (UID: \"f212310f-484e-4abf-a368-97e7193cc841\") "
Jul 27 20:00:39.806998 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.806970    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f212310f-484e-4abf-a368-97e7193cc841-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f212310f-484e-4abf-a368-97e7193cc841" (UID: "f212310f-484e-4abf-a368-97e7193cc841"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:00:39.808805 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.808780    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f212310f-484e-4abf-a368-97e7193cc841-kube-api-access-crpzc" (OuterVolumeSpecName: "kube-api-access-crpzc") pod "f212310f-484e-4abf-a368-97e7193cc841" (UID: "f212310f-484e-4abf-a368-97e7193cc841"). InnerVolumeSpecName "kube-api-access-crpzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:00:39.886195 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.886161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:39.907284 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.907198    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d478g\" (UniqueName: \"kubernetes.io/projected/00b65138-c029-44eb-aa63-08c822dc0cbc-kube-api-access-d478g\") pod \"00b65138-c029-44eb-aa63-08c822dc0cbc\" (UID: \"00b65138-c029-44eb-aa63-08c822dc0cbc\") "
Jul 27 20:00:39.907377 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.907286    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b65138-c029-44eb-aa63-08c822dc0cbc-plugins-config-volume\") pod \"00b65138-c029-44eb-aa63-08c822dc0cbc\" (UID: \"00b65138-c029-44eb-aa63-08c822dc0cbc\") "
Jul 27 20:00:39.907536 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.907523    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crpzc\" (UniqueName: \"kubernetes.io/projected/f212310f-484e-4abf-a368-97e7193cc841-kube-api-access-crpzc\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:00:39.907576 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.907537    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f212310f-484e-4abf-a368-97e7193cc841-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:00:39.907710 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.907680    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b65138-c029-44eb-aa63-08c822dc0cbc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00b65138-c029-44eb-aa63-08c822dc0cbc" (UID: "00b65138-c029-44eb-aa63-08c822dc0cbc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:00:39.909458 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:39.909439    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b65138-c029-44eb-aa63-08c822dc0cbc-kube-api-access-d478g" (OuterVolumeSpecName: "kube-api-access-d478g") pod "00b65138-c029-44eb-aa63-08c822dc0cbc" (UID: "00b65138-c029-44eb-aa63-08c822dc0cbc"). InnerVolumeSpecName "kube-api-access-d478g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:00:40.008219 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.008180    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b65138-c029-44eb-aa63-08c822dc0cbc-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:00:40.008219 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.008210    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d478g\" (UniqueName: \"kubernetes.io/projected/00b65138-c029-44eb-aa63-08c822dc0cbc-kube-api-access-d478g\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:00:40.278007 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.277976    2579 generic.go:356] "Generic (PLEG): container finished" podID="f212310f-484e-4abf-a368-97e7193cc841" containerID="d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f" exitCode=0
Jul 27 20:00:40.278219 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.278017    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"
Jul 27 20:00:40.278219 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.278059    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6" event={"ID":"f212310f-484e-4abf-a368-97e7193cc841","Type":"ContainerDied","Data":"d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f"}
Jul 27 20:00:40.278219 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.278114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6" event={"ID":"f212310f-484e-4abf-a368-97e7193cc841","Type":"ContainerDied","Data":"9adf93f4288d3d9d8b74496ac724ec410537571238a93e754d328307c1efa246"}
Jul 27 20:00:40.278219 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.278146    2579 scope.go:117] "RemoveContainer" containerID="d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f"
Jul 27 20:00:40.279485 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.279463    2579 generic.go:356] "Generic (PLEG): container finished" podID="00b65138-c029-44eb-aa63-08c822dc0cbc" containerID="8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31" exitCode=0
Jul 27 20:00:40.279593 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.279500    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"
Jul 27 20:00:40.279593 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.279520    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc" event={"ID":"00b65138-c029-44eb-aa63-08c822dc0cbc","Type":"ContainerDied","Data":"8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31"}
Jul 27 20:00:40.279593 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.279564    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc" event={"ID":"00b65138-c029-44eb-aa63-08c822dc0cbc","Type":"ContainerDied","Data":"71d12298e9d9f0f3a4e36a955cf9bfe5742e517950de2ef242574f6513e75efc"}
Jul 27 20:00:40.288237 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.288218    2579 scope.go:117] "RemoveContainer" containerID="d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f"
Jul 27 20:00:40.288578 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:00:40.288555    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f\": container with ID starting with d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f not found: ID does not exist" containerID="d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f"
Jul 27 20:00:40.288654 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.288590    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f"} err="failed to get container status \"d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f\": rpc error: code = NotFound desc = could not find container \"d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f\": container with ID starting with d57c937134fd65e4958045ff425c0d7f3de5abb197c47da8b0612ceaf4ea7f2f not found: ID does not exist"
Jul 27 20:00:40.288654 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.288615    2579 scope.go:117] "RemoveContainer" containerID="8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31"
Jul 27 20:00:40.298809 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.298790    2579 scope.go:117] "RemoveContainer" containerID="8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31"
Jul 27 20:00:40.299391 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:00:40.299362    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31\": container with ID starting with 8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31 not found: ID does not exist" containerID="8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31"
Jul 27 20:00:40.299485 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.299399    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31"} err="failed to get container status \"8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31\": rpc error: code = NotFound desc = could not find container \"8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31\": container with ID starting with 8dc24909f16438ee8f44bb6b50e6227fd6618a088efec3ce6f53810ee29cca31 not found: ID does not exist"
Jul 27 20:00:40.303109 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.303088    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"]
Jul 27 20:00:40.306658 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.306638    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-61128fd0-f4bf5fc4d-bhsnc"]
Jul 27 20:00:40.316996 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.316971    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"]
Jul 27 20:00:40.318738 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.318719    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-61128fd0-67c7955fd-xbvr6"]
Jul 27 20:00:40.489107 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.489074    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b65138-c029-44eb-aa63-08c822dc0cbc" path="/var/lib/kubelet/pods/00b65138-c029-44eb-aa63-08c822dc0cbc/volumes"
Jul 27 20:00:40.489523 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:40.489506    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f212310f-484e-4abf-a368-97e7193cc841" path="/var/lib/kubelet/pods/f212310f-484e-4abf-a368-97e7193cc841/volumes"
Jul 27 20:00:44.194669 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:44.194640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:44.295022 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:44.294988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:48.490881 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:48.490855    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:00:48.491992 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:48.491972    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:00:48.495001 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:48.494978    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:00:48.496018 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:48.495999    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:00:50.464733 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:50.464696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:50.564379 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:50.564346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:50.668534 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:50.668493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:50.767291 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:50.767192    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:50.874215 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:50.874181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:56.641901 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:56.641867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:56.701531 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:56.701499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:56.763780 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:56.763744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:56.865383 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:56.865352    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:00:56.985277 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:00:56.985219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:02.667955 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:02.667928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:02.728939 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:02.728911    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:09.378012 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:09.377977    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:09.646697 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:09.646611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:09.912596 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:09.912515    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:12.046286 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.046238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:12.144216 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.144183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:12.244136 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.244099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:12.552312 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.552277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:12.655897 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.655857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:12.731327 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.731295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:12.732166 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.732146    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"]
Jul 27 20:01:12.732585 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.732572    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b65138-c029-44eb-aa63-08c822dc0cbc" containerName="payload-pre-processing"
Jul 27 20:01:12.732640 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.732588    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b65138-c029-44eb-aa63-08c822dc0cbc" containerName="payload-pre-processing"
Jul 27 20:01:12.732640 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.732607    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f212310f-484e-4abf-a368-97e7193cc841" containerName="payload-processing"
Jul 27 20:01:12.732640 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.732613    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f212310f-484e-4abf-a368-97e7193cc841" containerName="payload-processing"
Jul 27 20:01:12.732739 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.732693    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b65138-c029-44eb-aa63-08c822dc0cbc" containerName="payload-pre-processing"
Jul 27 20:01:12.732739 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.732704    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f212310f-484e-4abf-a368-97e7193cc841" containerName="payload-processing"
Jul 27 20:01:12.736557 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.736538    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:12.738889 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.738871    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-5fa9bc6b\""
Jul 27 20:01:12.739029 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.738874    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-5fa9bc6b-dockercfg-5dhhd\""
Jul 27 20:01:12.744497 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.744469    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"]
Jul 27 20:01:12.755716 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.755689    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"]
Jul 27 20:01:12.759855 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.759832    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:12.768033 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.768010    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"]
Jul 27 20:01:12.821990 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.821896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxb9n\" (UniqueName: \"kubernetes.io/projected/55519b82-50fe-4810-8478-791fb92ae138-kube-api-access-fxb9n\") pod \"payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f\" (UID: \"55519b82-50fe-4810-8478-791fb92ae138\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:12.821990 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.821955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417bb1e3-e9df-4b8c-ae75-11b8167454a0-plugins-config-volume\") pod \"payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt\" (UID: \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\") " pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:12.822182 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.822015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhb4r\" (UniqueName: \"kubernetes.io/projected/417bb1e3-e9df-4b8c-ae75-11b8167454a0-kube-api-access-mhb4r\") pod \"payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt\" (UID: \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\") " pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:12.822182 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.822053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55519b82-50fe-4810-8478-791fb92ae138-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f\" (UID: \"55519b82-50fe-4810-8478-791fb92ae138\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:12.922676 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.922634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxb9n\" (UniqueName: \"kubernetes.io/projected/55519b82-50fe-4810-8478-791fb92ae138-kube-api-access-fxb9n\") pod \"payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f\" (UID: \"55519b82-50fe-4810-8478-791fb92ae138\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:12.922879 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.922685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417bb1e3-e9df-4b8c-ae75-11b8167454a0-plugins-config-volume\") pod \"payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt\" (UID: \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\") " pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:12.922879 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.922714    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhb4r\" (UniqueName: \"kubernetes.io/projected/417bb1e3-e9df-4b8c-ae75-11b8167454a0-kube-api-access-mhb4r\") pod \"payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt\" (UID: \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\") " pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:12.922879 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.922741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55519b82-50fe-4810-8478-791fb92ae138-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f\" (UID: \"55519b82-50fe-4810-8478-791fb92ae138\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:12.923657 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.923620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55519b82-50fe-4810-8478-791fb92ae138-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f\" (UID: \"55519b82-50fe-4810-8478-791fb92ae138\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:12.923781 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.923620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417bb1e3-e9df-4b8c-ae75-11b8167454a0-plugins-config-volume\") pod \"payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt\" (UID: \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\") " pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:12.930638 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.930614    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxb9n\" (UniqueName: \"kubernetes.io/projected/55519b82-50fe-4810-8478-791fb92ae138-kube-api-access-fxb9n\") pod \"payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f\" (UID: \"55519b82-50fe-4810-8478-791fb92ae138\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:12.930742 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:12.930683    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhb4r\" (UniqueName: \"kubernetes.io/projected/417bb1e3-e9df-4b8c-ae75-11b8167454a0-kube-api-access-mhb4r\") pod \"payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt\" (UID: \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\") " pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:13.049371 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.049339    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:13.070778 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.070751    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:13.199257 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.199144    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"]
Jul 27 20:01:13.202100 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:01:13.202059    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55519b82_50fe_4810_8478_791fb92ae138.slice/crio-f0c4e15ccfa01b440247f60e683c418178dcc65324c3307ecb27d8336b2cccec WatchSource:0}: Error finding container f0c4e15ccfa01b440247f60e683c418178dcc65324c3307ecb27d8336b2cccec: Status 404 returned error can't find the container with id f0c4e15ccfa01b440247f60e683c418178dcc65324c3307ecb27d8336b2cccec
Jul 27 20:01:13.226317 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.226269    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"]
Jul 27 20:01:13.231552 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:01:13.231525    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417bb1e3_e9df_4b8c_ae75_11b8167454a0.slice/crio-689f07f022a4de87a6de68014d96a4ed9ed1a889e075c56bab3591314bd6cb2b WatchSource:0}: Error finding container 689f07f022a4de87a6de68014d96a4ed9ed1a889e075c56bab3591314bd6cb2b: Status 404 returned error can't find the container with id 689f07f022a4de87a6de68014d96a4ed9ed1a889e075c56bab3591314bd6cb2b
Jul 27 20:01:13.461593 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.461522    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt" event={"ID":"417bb1e3-e9df-4b8c-ae75-11b8167454a0","Type":"ContainerStarted","Data":"035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc"}
Jul 27 20:01:13.461794 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.461605    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt" event={"ID":"417bb1e3-e9df-4b8c-ae75-11b8167454a0","Type":"ContainerStarted","Data":"689f07f022a4de87a6de68014d96a4ed9ed1a889e075c56bab3591314bd6cb2b"}
Jul 27 20:01:13.461794 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.461766    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:13.463264 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.463213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f" event={"ID":"55519b82-50fe-4810-8478-791fb92ae138","Type":"ContainerStarted","Data":"861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752"}
Jul 27 20:01:13.463392 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.463289    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:13.463392 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.463317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f" event={"ID":"55519b82-50fe-4810-8478-791fb92ae138","Type":"ContainerStarted","Data":"f0c4e15ccfa01b440247f60e683c418178dcc65324c3307ecb27d8336b2cccec"}
Jul 27 20:01:13.478718 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.478667    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt" podStartSLOduration=1.478648376 podStartE2EDuration="1.478648376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:01:13.477223601 +0000 UTC m=+2125.568575129" watchObservedRunningTime="2026-07-27 20:01:13.478648376 +0000 UTC m=+2125.569999873"
Jul 27 20:01:13.495496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:13.495446    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f" podStartSLOduration=1.495430131 podStartE2EDuration="1.495430131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:01:13.49361139 +0000 UTC m=+2125.584962889" watchObservedRunningTime="2026-07-27 20:01:13.495430131 +0000 UTC m=+2125.586781628"
Jul 27 20:01:16.535436 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:16.535398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:24.471255 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:24.471210    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:01:24.473626 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:24.471310    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:01:35.123671 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.123611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:35.196684 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.196642    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"]
Jul 27 20:01:35.200834 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.200809    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.205454 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.205428    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5fa9bc6b\"/\"default-dockercfg-rznrk\""
Jul 27 20:01:35.205810 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.205702    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5fa9bc6b\"/\"openshift-service-ca.crt\""
Jul 27 20:01:35.206017 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.205997    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5fa9bc6b\"/\"e2e-lifecycle-model-5fa9bc6b-kserve-self-signed-certs\""
Jul 27 20:01:35.206214 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.206009    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5fa9bc6b\"/\"kube-root-ca.crt\""
Jul 27 20:01:35.209013 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.208990    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"]
Jul 27 20:01:35.332917 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.332886    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rzfn\" (UniqueName: \"kubernetes.io/projected/4560a09a-12ce-45fb-9a77-046565539be1-kube-api-access-8rzfn\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.333077 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.332922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-dshm\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.333077 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.332987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-home\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.333077 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.333037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-model-cache\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.333077 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.333070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4560a09a-12ce-45fb-9a77-046565539be1-tls-certs\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.333222 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.333105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-tmp-dir\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434160 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-model-cache\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434370 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4560a09a-12ce-45fb-9a77-046565539be1-tls-certs\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434370 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-tmp-dir\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434370 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rzfn\" (UniqueName: \"kubernetes.io/projected/4560a09a-12ce-45fb-9a77-046565539be1-kube-api-access-8rzfn\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434551 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-dshm\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434551 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-home\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434670 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434640    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-model-cache\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434743 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-tmp-dir\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.434829 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.434790    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-home\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.437081 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.437052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-dshm\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.437200 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.437146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4560a09a-12ce-45fb-9a77-046565539be1-tls-certs\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.444775 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.444747    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rzfn\" (UniqueName: \"kubernetes.io/projected/4560a09a-12ce-45fb-9a77-046565539be1-kube-api-access-8rzfn\") pod \"e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") " pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.496185 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.496147    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:01:35.515187 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.515154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:35.652491 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:35.652465    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"]
Jul 27 20:01:35.655939 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:01:35.655907    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4560a09a_12ce_45fb_9a77_046565539be1.slice/crio-4fd4c14140ef3a5a7d21ebdd419093947ed0b115b2e02f64effeaed7730dabf2 WatchSource:0}: Error finding container 4fd4c14140ef3a5a7d21ebdd419093947ed0b115b2e02f64effeaed7730dabf2: Status 404 returned error can't find the container with id 4fd4c14140ef3a5a7d21ebdd419093947ed0b115b2e02f64effeaed7730dabf2
Jul 27 20:01:36.578869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:36.578836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr" event={"ID":"4560a09a-12ce-45fb-9a77-046565539be1","Type":"ContainerStarted","Data":"543f01007c8ec2ddfe91412c1ba821740ac1fbaa55d24f150651dee3b67d36f7"}
Jul 27 20:01:36.578869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:36.578876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr" event={"ID":"4560a09a-12ce-45fb-9a77-046565539be1","Type":"ContainerStarted","Data":"4fd4c14140ef3a5a7d21ebdd419093947ed0b115b2e02f64effeaed7730dabf2"}
Jul 27 20:01:36.595920 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:36.595874    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr" podStartSLOduration=1.595857254 podStartE2EDuration="1.595857254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:01:36.594602215 +0000 UTC m=+2148.685953718" watchObservedRunningTime="2026-07-27 20:01:36.595857254 +0000 UTC m=+2148.687208743"
Jul 27 20:01:45.515506 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:45.515413    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:45.515506 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:45.515470    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:45.528127 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:45.528096    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:01:45.634443 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:01:45.634417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:02:21.860169 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:21.860132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:21.883577 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:21.883544    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"]
Jul 27 20:02:21.883821 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:21.883774    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt" podUID="417bb1e3-e9df-4b8c-ae75-11b8167454a0" containerName="payload-processing" containerID="cri-o://035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc" gracePeriod=30
Jul 27 20:02:21.892752 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:21.892722    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"]
Jul 27 20:02:21.892994 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:21.892969    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f" podUID="55519b82-50fe-4810-8478-791fb92ae138" containerName="payload-pre-processing" containerID="cri-o://861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752" gracePeriod=30
Jul 27 20:02:21.959991 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:21.959962    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:22.060656 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.060622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:22.173365 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.173341    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:02:22.176339 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.176317    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:02:22.181739 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.181722    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417bb1e3-e9df-4b8c-ae75-11b8167454a0-plugins-config-volume\") pod \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\" (UID: \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\") "
Jul 27 20:02:22.181818 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.181789    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhb4r\" (UniqueName: \"kubernetes.io/projected/417bb1e3-e9df-4b8c-ae75-11b8167454a0-kube-api-access-mhb4r\") pod \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\" (UID: \"417bb1e3-e9df-4b8c-ae75-11b8167454a0\") "
Jul 27 20:02:22.182092 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.182069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/417bb1e3-e9df-4b8c-ae75-11b8167454a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "417bb1e3-e9df-4b8c-ae75-11b8167454a0" (UID: "417bb1e3-e9df-4b8c-ae75-11b8167454a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:02:22.183791 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.183769    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/417bb1e3-e9df-4b8c-ae75-11b8167454a0-kube-api-access-mhb4r" (OuterVolumeSpecName: "kube-api-access-mhb4r") pod "417bb1e3-e9df-4b8c-ae75-11b8167454a0" (UID: "417bb1e3-e9df-4b8c-ae75-11b8167454a0"). InnerVolumeSpecName "kube-api-access-mhb4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:02:22.283318 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.283235    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxb9n\" (UniqueName: \"kubernetes.io/projected/55519b82-50fe-4810-8478-791fb92ae138-kube-api-access-fxb9n\") pod \"55519b82-50fe-4810-8478-791fb92ae138\" (UID: \"55519b82-50fe-4810-8478-791fb92ae138\") "
Jul 27 20:02:22.283504 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.283340    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55519b82-50fe-4810-8478-791fb92ae138-plugins-config-volume\") pod \"55519b82-50fe-4810-8478-791fb92ae138\" (UID: \"55519b82-50fe-4810-8478-791fb92ae138\") "
Jul 27 20:02:22.283562 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.283548    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417bb1e3-e9df-4b8c-ae75-11b8167454a0-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:22.283604 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.283564    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhb4r\" (UniqueName: \"kubernetes.io/projected/417bb1e3-e9df-4b8c-ae75-11b8167454a0-kube-api-access-mhb4r\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:22.283767 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.283744    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55519b82-50fe-4810-8478-791fb92ae138-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55519b82-50fe-4810-8478-791fb92ae138" (UID: "55519b82-50fe-4810-8478-791fb92ae138"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:02:22.285487 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.285467    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55519b82-50fe-4810-8478-791fb92ae138-kube-api-access-fxb9n" (OuterVolumeSpecName: "kube-api-access-fxb9n") pod "55519b82-50fe-4810-8478-791fb92ae138" (UID: "55519b82-50fe-4810-8478-791fb92ae138"). InnerVolumeSpecName "kube-api-access-fxb9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:02:22.384575 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.384522    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxb9n\" (UniqueName: \"kubernetes.io/projected/55519b82-50fe-4810-8478-791fb92ae138-kube-api-access-fxb9n\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:22.384575 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.384568    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55519b82-50fe-4810-8478-791fb92ae138-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:22.809106 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.809016    2579 generic.go:356] "Generic (PLEG): container finished" podID="417bb1e3-e9df-4b8c-ae75-11b8167454a0" containerID="035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc" exitCode=0
Jul 27 20:02:22.809106 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.809064    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"
Jul 27 20:02:22.809364 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.809111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt" event={"ID":"417bb1e3-e9df-4b8c-ae75-11b8167454a0","Type":"ContainerDied","Data":"035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc"}
Jul 27 20:02:22.809364 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.809161    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt" event={"ID":"417bb1e3-e9df-4b8c-ae75-11b8167454a0","Type":"ContainerDied","Data":"689f07f022a4de87a6de68014d96a4ed9ed1a889e075c56bab3591314bd6cb2b"}
Jul 27 20:02:22.809364 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.809183    2579 scope.go:117] "RemoveContainer" containerID="035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc"
Jul 27 20:02:22.810449 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.810422    2579 generic.go:356] "Generic (PLEG): container finished" podID="55519b82-50fe-4810-8478-791fb92ae138" containerID="861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752" exitCode=0
Jul 27 20:02:22.810567 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.810459    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"
Jul 27 20:02:22.810567 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.810491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f" event={"ID":"55519b82-50fe-4810-8478-791fb92ae138","Type":"ContainerDied","Data":"861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752"}
Jul 27 20:02:22.810567 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.810534    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f" event={"ID":"55519b82-50fe-4810-8478-791fb92ae138","Type":"ContainerDied","Data":"f0c4e15ccfa01b440247f60e683c418178dcc65324c3307ecb27d8336b2cccec"}
Jul 27 20:02:22.818054 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.818037    2579 scope.go:117] "RemoveContainer" containerID="035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc"
Jul 27 20:02:22.818389 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:02:22.818363    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc\": container with ID starting with 035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc not found: ID does not exist" containerID="035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc"
Jul 27 20:02:22.818469 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.818398    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc"} err="failed to get container status \"035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc\": rpc error: code = NotFound desc = could not find container \"035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc\": container with ID starting with 035113ebdfc2f896f988fe6cf770c09703a9181acba6a9feaa5c3d42fdfc2cdc not found: ID does not exist"
Jul 27 20:02:22.818469 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.818417    2579 scope.go:117] "RemoveContainer" containerID="861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752"
Jul 27 20:02:22.826483 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.826412    2579 scope.go:117] "RemoveContainer" containerID="861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752"
Jul 27 20:02:22.826723 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:02:22.826702    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752\": container with ID starting with 861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752 not found: ID does not exist" containerID="861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752"
Jul 27 20:02:22.826800 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.826729    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752"} err="failed to get container status \"861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752\": rpc error: code = NotFound desc = could not find container \"861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752\": container with ID starting with 861bd1331236c076fdcb238afec40df4b1fff8b907c96011f6f8b6a0869f0752 not found: ID does not exist"
Jul 27 20:02:22.828383 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.828361    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"]
Jul 27 20:02:22.831598 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.831578    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5fa9bc6b-68489d9c48-qzv2f"]
Jul 27 20:02:22.841986 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.841959    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"]
Jul 27 20:02:22.843663 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:22.843643    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5fa9bc6b-6945948fcd-6nmgt"]
Jul 27 20:02:24.488499 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:24.488466    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="417bb1e3-e9df-4b8c-ae75-11b8167454a0" path="/var/lib/kubelet/pods/417bb1e3-e9df-4b8c-ae75-11b8167454a0/volumes"
Jul 27 20:02:24.488876 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:24.488856    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55519b82-50fe-4810-8478-791fb92ae138" path="/var/lib/kubelet/pods/55519b82-50fe-4810-8478-791fb92ae138/volumes"
Jul 27 20:02:27.056527 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:27.056497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:30.357398 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:30.357369    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:34.735495 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:34.735461    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"]
Jul 27 20:02:34.791180 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:34.791148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:40.874798 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:40.874758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:02:48.600469 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:48.600438    2579 scope.go:117] "RemoveContainer" containerID="4b5625973beae60bac42220c98b53b5ca1acaf1b0f9693926e738f0e0ea02c79"
Jul 27 20:02:49.755067 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:49.754993    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr" podUID="4560a09a-12ce-45fb-9a77-046565539be1" containerName="main" containerID="cri-o://543f01007c8ec2ddfe91412c1ba821740ac1fbaa55d24f150651dee3b67d36f7" gracePeriod=45
Jul 27 20:02:49.950949 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:49.950918    2579 generic.go:356] "Generic (PLEG): container finished" podID="4560a09a-12ce-45fb-9a77-046565539be1" containerID="543f01007c8ec2ddfe91412c1ba821740ac1fbaa55d24f150651dee3b67d36f7" exitCode=0
Jul 27 20:02:49.951103 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:49.950993    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr" event={"ID":"4560a09a-12ce-45fb-9a77-046565539be1","Type":"ContainerDied","Data":"543f01007c8ec2ddfe91412c1ba821740ac1fbaa55d24f150651dee3b67d36f7"}
Jul 27 20:02:50.005339 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.005278    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:02:50.146083 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146051    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rzfn\" (UniqueName: \"kubernetes.io/projected/4560a09a-12ce-45fb-9a77-046565539be1-kube-api-access-8rzfn\") pod \"4560a09a-12ce-45fb-9a77-046565539be1\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") "
Jul 27 20:02:50.146297 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-dshm\") pod \"4560a09a-12ce-45fb-9a77-046565539be1\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") "
Jul 27 20:02:50.146297 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146147    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-tmp-dir\") pod \"4560a09a-12ce-45fb-9a77-046565539be1\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") "
Jul 27 20:02:50.146297 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146239    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-model-cache\") pod \"4560a09a-12ce-45fb-9a77-046565539be1\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") "
Jul 27 20:02:50.146495 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146309    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4560a09a-12ce-45fb-9a77-046565539be1-tls-certs\") pod \"4560a09a-12ce-45fb-9a77-046565539be1\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") "
Jul 27 20:02:50.146495 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146340    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-home\") pod \"4560a09a-12ce-45fb-9a77-046565539be1\" (UID: \"4560a09a-12ce-45fb-9a77-046565539be1\") "
Jul 27 20:02:50.146495 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146475    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-model-cache" (OuterVolumeSpecName: "model-cache") pod "4560a09a-12ce-45fb-9a77-046565539be1" (UID: "4560a09a-12ce-45fb-9a77-046565539be1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:50.146640 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146489    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4560a09a-12ce-45fb-9a77-046565539be1" (UID: "4560a09a-12ce-45fb-9a77-046565539be1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:50.146693 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146663    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-home" (OuterVolumeSpecName: "home") pod "4560a09a-12ce-45fb-9a77-046565539be1" (UID: "4560a09a-12ce-45fb-9a77-046565539be1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:50.146756 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146741    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-model-cache\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:50.146812 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.146788    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-tmp-dir\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:50.148447 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.148420    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-dshm" (OuterVolumeSpecName: "dshm") pod "4560a09a-12ce-45fb-9a77-046565539be1" (UID: "4560a09a-12ce-45fb-9a77-046565539be1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:50.148556 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.148454    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4560a09a-12ce-45fb-9a77-046565539be1-kube-api-access-8rzfn" (OuterVolumeSpecName: "kube-api-access-8rzfn") pod "4560a09a-12ce-45fb-9a77-046565539be1" (UID: "4560a09a-12ce-45fb-9a77-046565539be1"). InnerVolumeSpecName "kube-api-access-8rzfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:02:50.148556 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.148485    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4560a09a-12ce-45fb-9a77-046565539be1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4560a09a-12ce-45fb-9a77-046565539be1" (UID: "4560a09a-12ce-45fb-9a77-046565539be1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:02:50.247909 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.247865    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4560a09a-12ce-45fb-9a77-046565539be1-tls-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:50.247909 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.247899    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-home\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:50.247909 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.247913    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rzfn\" (UniqueName: \"kubernetes.io/projected/4560a09a-12ce-45fb-9a77-046565539be1-kube-api-access-8rzfn\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:50.248160 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.247924    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4560a09a-12ce-45fb-9a77-046565539be1-dshm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:02:50.956790 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.956763    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"
Jul 27 20:02:50.956790 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.956773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr" event={"ID":"4560a09a-12ce-45fb-9a77-046565539be1","Type":"ContainerDied","Data":"4fd4c14140ef3a5a7d21ebdd419093947ed0b115b2e02f64effeaed7730dabf2"}
Jul 27 20:02:50.957282 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.956817    2579 scope.go:117] "RemoveContainer" containerID="543f01007c8ec2ddfe91412c1ba821740ac1fbaa55d24f150651dee3b67d36f7"
Jul 27 20:02:50.973439 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.973416    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"]
Jul 27 20:02:50.975181 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:50.975157    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5fa9bc6b/e2e-lifecycle-model-5fa9bc6b-kserve-76976484b7-qlsjr"]
Jul 27 20:02:52.488603 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:02:52.488571    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4560a09a-12ce-45fb-9a77-046565539be1" path="/var/lib/kubelet/pods/4560a09a-12ce-45fb-9a77-046565539be1/volumes"
Jul 27 20:03:11.046375 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:11.046338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:12.407590 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:12.407555    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:17.964517 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:17.964483    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:18.064805 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:18.064770    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:18.233172 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:18.233087    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:18.493165 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:18.493087    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:19.486966 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:19.486933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:19.585939 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:19.585910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:19.686302 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:19.686260    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:20.379590 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:20.379563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:20.483683 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:20.483650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:20.582133 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:20.582102    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:21.908458 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:21.908425    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:27.760730 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:27.760695    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:27.834662 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:27.834624    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:28.478767 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:28.478731    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:35.179098 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:35.179062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:35.282332 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:35.282290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:35.400770 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:35.400733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:45.883086 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:45.883048    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:46.229712 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:46.229675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:03:48.538593 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:03:48.538563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:07.129460 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:07.129423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:11.291821 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:11.291792    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:11.392867 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:11.392832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:13.212143 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:13.212112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:13.312269 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:13.312190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:13.553428 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:13.553321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:13.844831 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:13.844744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:14.020680 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:14.020643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:17.029620 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.029585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:17.204892 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.204859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:17.380876 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.380781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:17.489037 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.489003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:17.767695 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.767662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:17.829852 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.829818    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"]
Jul 27 20:04:17.830218 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830206    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55519b82-50fe-4810-8478-791fb92ae138" containerName="payload-pre-processing"
Jul 27 20:04:17.830290 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830219    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="55519b82-50fe-4810-8478-791fb92ae138" containerName="payload-pre-processing"
Jul 27 20:04:17.830290 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830227    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4560a09a-12ce-45fb-9a77-046565539be1" containerName="main"
Jul 27 20:04:17.830290 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830233    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4560a09a-12ce-45fb-9a77-046565539be1" containerName="main"
Jul 27 20:04:17.830290 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830271    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="417bb1e3-e9df-4b8c-ae75-11b8167454a0" containerName="payload-processing"
Jul 27 20:04:17.830290 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830278    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="417bb1e3-e9df-4b8c-ae75-11b8167454a0" containerName="payload-processing"
Jul 27 20:04:17.830450 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830336    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="417bb1e3-e9df-4b8c-ae75-11b8167454a0" containerName="payload-processing"
Jul 27 20:04:17.830450 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830345    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4560a09a-12ce-45fb-9a77-046565539be1" containerName="main"
Jul 27 20:04:17.830450 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.830354    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="55519b82-50fe-4810-8478-791fb92ae138" containerName="payload-pre-processing"
Jul 27 20:04:17.833392 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.833371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:17.835894 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.835873    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-20b70e\""
Jul 27 20:04:17.840717 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.840689    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"]
Jul 27 20:04:17.844932 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.844908    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:17.863452 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.863417    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"]
Jul 27 20:04:17.867918 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.867894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:17.870782 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.870758    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-20b70e-dockercfg-sdg5h\""
Jul 27 20:04:17.873869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.873845    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"]
Jul 27 20:04:17.961346 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.961311    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0ec0c9-8d78-4727-938c-582c7df21a42-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s\" (UID: \"cc0ec0c9-8d78-4727-938c-582c7df21a42\") " pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:17.961525 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.961445    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7zzr\" (UniqueName: \"kubernetes.io/projected/3f37476f-34f8-421f-b16c-4d38261bd3fd-kube-api-access-b7zzr\") pod \"payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx\" (UID: \"3f37476f-34f8-421f-b16c-4d38261bd3fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:17.961525 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.961507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcdtt\" (UniqueName: \"kubernetes.io/projected/cc0ec0c9-8d78-4727-938c-582c7df21a42-kube-api-access-lcdtt\") pod \"payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s\" (UID: \"cc0ec0c9-8d78-4727-938c-582c7df21a42\") " pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:17.961602 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:17.961586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f37476f-34f8-421f-b16c-4d38261bd3fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx\" (UID: \"3f37476f-34f8-421f-b16c-4d38261bd3fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:18.062454 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.062365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7zzr\" (UniqueName: \"kubernetes.io/projected/3f37476f-34f8-421f-b16c-4d38261bd3fd-kube-api-access-b7zzr\") pod \"payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx\" (UID: \"3f37476f-34f8-421f-b16c-4d38261bd3fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:18.062454 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.062408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdtt\" (UniqueName: \"kubernetes.io/projected/cc0ec0c9-8d78-4727-938c-582c7df21a42-kube-api-access-lcdtt\") pod \"payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s\" (UID: \"cc0ec0c9-8d78-4727-938c-582c7df21a42\") " pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:18.063002 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.062459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f37476f-34f8-421f-b16c-4d38261bd3fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx\" (UID: \"3f37476f-34f8-421f-b16c-4d38261bd3fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:18.063002 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.062485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0ec0c9-8d78-4727-938c-582c7df21a42-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s\" (UID: \"cc0ec0c9-8d78-4727-938c-582c7df21a42\") " pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:18.063279 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.063226    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0ec0c9-8d78-4727-938c-582c7df21a42-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s\" (UID: \"cc0ec0c9-8d78-4727-938c-582c7df21a42\") " pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:18.063378 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.063335    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f37476f-34f8-421f-b16c-4d38261bd3fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx\" (UID: \"3f37476f-34f8-421f-b16c-4d38261bd3fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:18.072174 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.072143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7zzr\" (UniqueName: \"kubernetes.io/projected/3f37476f-34f8-421f-b16c-4d38261bd3fd-kube-api-access-b7zzr\") pod \"payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx\" (UID: \"3f37476f-34f8-421f-b16c-4d38261bd3fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:18.072319 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.072143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcdtt\" (UniqueName: \"kubernetes.io/projected/cc0ec0c9-8d78-4727-938c-582c7df21a42-kube-api-access-lcdtt\") pod \"payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s\" (UID: \"cc0ec0c9-8d78-4727-938c-582c7df21a42\") " pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:18.145836 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.145803    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:18.181364 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.181335    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:18.291933 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.291905    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"]
Jul 27 20:04:18.293618 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:04:18.293591    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f37476f_34f8_421f_b16c_4d38261bd3fd.slice/crio-fdc1a29f107089dc9308c13b9a691839b8c74e978f289fe65be4e6546f40ab58 WatchSource:0}: Error finding container fdc1a29f107089dc9308c13b9a691839b8c74e978f289fe65be4e6546f40ab58: Status 404 returned error can't find the container with id fdc1a29f107089dc9308c13b9a691839b8c74e978f289fe65be4e6546f40ab58
Jul 27 20:04:18.328936 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.328915    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"]
Jul 27 20:04:18.331370 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:04:18.331344    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0ec0c9_8d78_4727_938c_582c7df21a42.slice/crio-fb8d763656878fb3e8160e76eda394bc84218b65c5106ab1c0739e17fac93680 WatchSource:0}: Error finding container fb8d763656878fb3e8160e76eda394bc84218b65c5106ab1c0739e17fac93680: Status 404 returned error can't find the container with id fb8d763656878fb3e8160e76eda394bc84218b65c5106ab1c0739e17fac93680
Jul 27 20:04:18.391729 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.391699    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx" event={"ID":"3f37476f-34f8-421f-b16c-4d38261bd3fd","Type":"ContainerStarted","Data":"8ff689fca0639cb7d4032e7b61c6215596e2ffb738945f19007c396aa3a51eff"}
Jul 27 20:04:18.391857 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.391763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx" event={"ID":"3f37476f-34f8-421f-b16c-4d38261bd3fd","Type":"ContainerStarted","Data":"fdc1a29f107089dc9308c13b9a691839b8c74e978f289fe65be4e6546f40ab58"}
Jul 27 20:04:18.391929 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.391905    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:18.393593 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.393566    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s" event={"ID":"cc0ec0c9-8d78-4727-938c-582c7df21a42","Type":"ContainerStarted","Data":"66ea0728675bf6435b99bd25e4afbaebc257f35653d5dd65b4d82e5d8d1d2686"}
Jul 27 20:04:18.393700 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.393623    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:18.393700 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.393648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s" event={"ID":"cc0ec0c9-8d78-4727-938c-582c7df21a42","Type":"ContainerStarted","Data":"fb8d763656878fb3e8160e76eda394bc84218b65c5106ab1c0739e17fac93680"}
Jul 27 20:04:18.407314 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.407261    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx" podStartSLOduration=1.407225588 podStartE2EDuration="1.407225588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:04:18.406563296 +0000 UTC m=+2310.497914792" watchObservedRunningTime="2026-07-27 20:04:18.407225588 +0000 UTC m=+2310.498577087"
Jul 27 20:04:18.424403 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:18.424361    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s" podStartSLOduration=1.424345839 podStartE2EDuration="1.424345839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:04:18.42334448 +0000 UTC m=+2310.514695977" watchObservedRunningTime="2026-07-27 20:04:18.424345839 +0000 UTC m=+2310.515697337"
Jul 27 20:04:21.516744 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:21.516703    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:21.846071 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:21.845968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:22.137463 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:22.137371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:22.218445 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:22.218404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:25.094107 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.094067    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:25.195485 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.195448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:25.295311 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.295272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:25.393822 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.393744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:25.745725 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.745694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:25.837534 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.837474    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:25.840880 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.840854    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"]
Jul 27 20:04:25.844557 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.844542    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:25.846931 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.846911    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-461c95-dockercfg-2knzx\""
Jul 27 20:04:25.847041 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.846915    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-461c95\""
Jul 27 20:04:25.853929 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.853906    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"]
Jul 27 20:04:25.937191 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.937119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mzjz\" (UniqueName: \"kubernetes.io/projected/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-kube-api-access-8mzjz\") pod \"payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b\" (UID: \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:25.937390 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:25.937282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b\" (UID: \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:26.038865 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:26.038774    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mzjz\" (UniqueName: \"kubernetes.io/projected/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-kube-api-access-8mzjz\") pod \"payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b\" (UID: \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:26.039051 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:26.038866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b\" (UID: \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:26.039610 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:26.039589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b\" (UID: \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:26.047207 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:26.047184    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mzjz\" (UniqueName: \"kubernetes.io/projected/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-kube-api-access-8mzjz\") pod \"payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b\" (UID: \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:26.157004 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:26.156962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:26.502227 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:26.502200    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"]
Jul 27 20:04:26.504055 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:04:26.504029    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed289c2_3d82_4cc9_b039_a29e77f9b1b3.slice/crio-060cbfea2633f55c8e4c5adf1d5f083c334e8dd588c8b969f713e7a7946959c4 WatchSource:0}: Error finding container 060cbfea2633f55c8e4c5adf1d5f083c334e8dd588c8b969f713e7a7946959c4: Status 404 returned error can't find the container with id 060cbfea2633f55c8e4c5adf1d5f083c334e8dd588c8b969f713e7a7946959c4
Jul 27 20:04:27.439452 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:27.439419    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b" event={"ID":"fed289c2-3d82-4cc9-b039-a29e77f9b1b3","Type":"ContainerStarted","Data":"0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32"}
Jul 27 20:04:27.439935 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:27.439474    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b" event={"ID":"fed289c2-3d82-4cc9-b039-a29e77f9b1b3","Type":"ContainerStarted","Data":"060cbfea2633f55c8e4c5adf1d5f083c334e8dd588c8b969f713e7a7946959c4"}
Jul 27 20:04:27.439935 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:27.439535    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:27.455493 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:27.455447    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b" podStartSLOduration=2.455431709 podStartE2EDuration="2.455431709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:04:27.454357705 +0000 UTC m=+2319.545709204" watchObservedRunningTime="2026-07-27 20:04:27.455431709 +0000 UTC m=+2319.546783206"
Jul 27 20:04:29.400443 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:29.400409    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:04:29.400843 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:29.400591    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:04:29.572453 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:29.572427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:38.446459 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:38.446428    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:04:49.591654 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.591574    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"]
Jul 27 20:04:49.596340 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.596321    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.599093 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.599071    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-20b70e\"/\"kube-root-ca.crt\""
Jul 27 20:04:49.600217 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.600195    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-20b70e\"/\"default-dockercfg-txzlw\""
Jul 27 20:04:49.600337 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.600267    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-20b70e\"/\"openshift-service-ca.crt\""
Jul 27 20:04:49.600337 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.600314    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-20b70e\"/\"test-model-20b70e-kserve-self-signed-certs\""
Jul 27 20:04:49.604717 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.604694    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"]
Jul 27 20:04:49.694983 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.694953    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:04:49.770592 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.770556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjxzw\" (UniqueName: \"kubernetes.io/projected/bafee261-bdae-485d-8edf-9506a788b468-kube-api-access-kjxzw\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.770759 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.770602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-model-cache\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.770759 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.770632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-tmp-dir\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.770759 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.770736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bafee261-bdae-485d-8edf-9506a788b468-tls-certs\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.770874 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.770789    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-home\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.770874 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.770813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-dshm\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871419 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjxzw\" (UniqueName: \"kubernetes.io/projected/bafee261-bdae-485d-8edf-9506a788b468-kube-api-access-kjxzw\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871419 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871377    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-model-cache\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871419 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-tmp-dir\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871675 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bafee261-bdae-485d-8edf-9506a788b468-tls-certs\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871675 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-home\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871675 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-dshm\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871892 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871871    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-tmp-dir\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-model-cache\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.871930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.871921    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-home\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.873780 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.873753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-dshm\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.874028 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.874010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bafee261-bdae-485d-8edf-9506a788b468-tls-certs\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.878958 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.878935    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjxzw\" (UniqueName: \"kubernetes.io/projected/bafee261-bdae-485d-8edf-9506a788b468-kube-api-access-kjxzw\") pod \"test-model-20b70e-kserve-754d5c8965-gdmhd\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") " pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:49.909000 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:49.908968    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:50.042751 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:50.042727    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"]
Jul 27 20:04:50.044990 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:04:50.044965    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbafee261_bdae_485d_8edf_9506a788b468.slice/crio-fcf23341169c9d639358d45c6d158ed213b938c8cf004b03390e8a22fda5e036 WatchSource:0}: Error finding container fcf23341169c9d639358d45c6d158ed213b938c8cf004b03390e8a22fda5e036: Status 404 returned error can't find the container with id fcf23341169c9d639358d45c6d158ed213b938c8cf004b03390e8a22fda5e036
Jul 27 20:04:50.561685 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:50.561649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd" event={"ID":"bafee261-bdae-485d-8edf-9506a788b468","Type":"ContainerStarted","Data":"6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5"}
Jul 27 20:04:50.561685 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:50.561688    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd" event={"ID":"bafee261-bdae-485d-8edf-9506a788b468","Type":"ContainerStarted","Data":"fcf23341169c9d639358d45c6d158ed213b938c8cf004b03390e8a22fda5e036"}
Jul 27 20:04:50.578688 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:50.578635    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd" podStartSLOduration=1.578618708 podStartE2EDuration="1.578618708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:04:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:04:50.577235659 +0000 UTC m=+2342.668587158" watchObservedRunningTime="2026-07-27 20:04:50.578618708 +0000 UTC m=+2342.669970205"
Jul 27 20:04:59.909420 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:59.909386    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:59.909833 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:59.909576    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:04:59.922097 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:04:59.922068    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:05:00.621665 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:00.621635    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:05:09.732645 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:09.732609    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:29.298631 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:29.298592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:40.660816 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:40.660733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:40.760550 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:40.760517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:05:48.523455 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:48.523426    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:05:48.524572 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:48.524548    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:05:48.529503 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:48.529484    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:05:48.530495 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:48.530478    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:05:59.270467 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:59.270423    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"]
Jul 27 20:05:59.289799 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:05:59.289766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:00.138030 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.137992    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:00.260132 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.260096    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"]
Jul 27 20:06:00.263835 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.263818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"
Jul 27 20:06:00.266235 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.266217    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 20:06:00.271501 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.271476    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"]
Jul 27 20:06:00.356621 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.356590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39510c1c-89f7-4938-b72b-f9c5ab3be12b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh\" (UID: \"39510c1c-89f7-4938-b72b-f9c5ab3be12b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"
Jul 27 20:06:00.457868 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.457826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39510c1c-89f7-4938-b72b-f9c5ab3be12b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh\" (UID: \"39510c1c-89f7-4938-b72b-f9c5ab3be12b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"
Jul 27 20:06:00.458466 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.458448    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39510c1c-89f7-4938-b72b-f9c5ab3be12b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh\" (UID: \"39510c1c-89f7-4938-b72b-f9c5ab3be12b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"
Jul 27 20:06:00.577519 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.577473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"
Jul 27 20:06:00.711055 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.711033    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"]
Jul 27 20:06:00.712842 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:06:00.712815    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39510c1c_89f7_4938_b72b_f9c5ab3be12b.slice/crio-f9f6a190031ff4a20bbcc9e8c860bee4192cf07385d071b5fbcaebd76a3a187c WatchSource:0}: Error finding container f9f6a190031ff4a20bbcc9e8c860bee4192cf07385d071b5fbcaebd76a3a187c: Status 404 returned error can't find the container with id f9f6a190031ff4a20bbcc9e8c860bee4192cf07385d071b5fbcaebd76a3a187c
Jul 27 20:06:00.912568 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:00.912530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh" event={"ID":"39510c1c-89f7-4938-b72b-f9c5ab3be12b","Type":"ContainerStarted","Data":"f9f6a190031ff4a20bbcc9e8c860bee4192cf07385d071b5fbcaebd76a3a187c"}
Jul 27 20:06:01.919142 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:01.919053    2579 generic.go:356] "Generic (PLEG): container finished" podID="39510c1c-89f7-4938-b72b-f9c5ab3be12b" containerID="54b1945801c4bcf0b315da12667aef0412c3fa4f9265023dae216e99151a9a59" exitCode=0
Jul 27 20:06:01.919510 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:01.919137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh" event={"ID":"39510c1c-89f7-4938-b72b-f9c5ab3be12b","Type":"ContainerDied","Data":"54b1945801c4bcf0b315da12667aef0412c3fa4f9265023dae216e99151a9a59"}
Jul 27 20:06:03.059786 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:03.059762    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"
Jul 27 20:06:03.186627 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:03.186599    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39510c1c-89f7-4938-b72b-f9c5ab3be12b-maas-api-service-ca\") pod \"39510c1c-89f7-4938-b72b-f9c5ab3be12b\" (UID: \"39510c1c-89f7-4938-b72b-f9c5ab3be12b\") "
Jul 27 20:06:03.186948 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:03.186927    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39510c1c-89f7-4938-b72b-f9c5ab3be12b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "39510c1c-89f7-4938-b72b-f9c5ab3be12b" (UID: "39510c1c-89f7-4938-b72b-f9c5ab3be12b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:03.287911 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:03.287877    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39510c1c-89f7-4938-b72b-f9c5ab3be12b-maas-api-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:03.931728 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:03.931703    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"
Jul 27 20:06:03.931728 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:03.931725    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh" event={"ID":"39510c1c-89f7-4938-b72b-f9c5ab3be12b","Type":"ContainerDied","Data":"f9f6a190031ff4a20bbcc9e8c860bee4192cf07385d071b5fbcaebd76a3a187c"}
Jul 27 20:06:03.931959 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:03.931774    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9f6a190031ff4a20bbcc9e8c860bee4192cf07385d071b5fbcaebd76a3a187c"
Jul 27 20:06:10.694437 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.694395    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:10.715455 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.715421    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"]
Jul 27 20:06:10.715738 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.715712    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s" podUID="cc0ec0c9-8d78-4727-938c-582c7df21a42" containerName="payload-processing" containerID="cri-o://66ea0728675bf6435b99bd25e4afbaebc257f35653d5dd65b4d82e5d8d1d2686" gracePeriod=30
Jul 27 20:06:10.720802 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.720774    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"]
Jul 27 20:06:10.721075 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.721024    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx" podUID="3f37476f-34f8-421f-b16c-4d38261bd3fd" containerName="payload-pre-processing" containerID="cri-o://8ff689fca0639cb7d4032e7b61c6215596e2ffb738945f19007c396aa3a51eff" gracePeriod=30
Jul 27 20:06:10.773048 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.772990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:10.970315 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.970282    2579 generic.go:356] "Generic (PLEG): container finished" podID="3f37476f-34f8-421f-b16c-4d38261bd3fd" containerID="8ff689fca0639cb7d4032e7b61c6215596e2ffb738945f19007c396aa3a51eff" exitCode=0
Jul 27 20:06:10.970465 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.970350    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx" event={"ID":"3f37476f-34f8-421f-b16c-4d38261bd3fd","Type":"ContainerDied","Data":"8ff689fca0639cb7d4032e7b61c6215596e2ffb738945f19007c396aa3a51eff"}
Jul 27 20:06:10.971684 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.971654    2579 generic.go:356] "Generic (PLEG): container finished" podID="cc0ec0c9-8d78-4727-938c-582c7df21a42" containerID="66ea0728675bf6435b99bd25e4afbaebc257f35653d5dd65b4d82e5d8d1d2686" exitCode=0
Jul 27 20:06:10.971821 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.971730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s" event={"ID":"cc0ec0c9-8d78-4727-938c-582c7df21a42","Type":"ContainerDied","Data":"66ea0728675bf6435b99bd25e4afbaebc257f35653d5dd65b4d82e5d8d1d2686"}
Jul 27 20:06:10.997992 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:10.997971    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:06:11.001287 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.001262    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:06:11.067768 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.067737    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7zzr\" (UniqueName: \"kubernetes.io/projected/3f37476f-34f8-421f-b16c-4d38261bd3fd-kube-api-access-b7zzr\") pod \"3f37476f-34f8-421f-b16c-4d38261bd3fd\" (UID: \"3f37476f-34f8-421f-b16c-4d38261bd3fd\") "
Jul 27 20:06:11.067768 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.067781    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f37476f-34f8-421f-b16c-4d38261bd3fd-plugins-config-volume\") pod \"3f37476f-34f8-421f-b16c-4d38261bd3fd\" (UID: \"3f37476f-34f8-421f-b16c-4d38261bd3fd\") "
Jul 27 20:06:11.067997 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.067802    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0ec0c9-8d78-4727-938c-582c7df21a42-plugins-config-volume\") pod \"cc0ec0c9-8d78-4727-938c-582c7df21a42\" (UID: \"cc0ec0c9-8d78-4727-938c-582c7df21a42\") "
Jul 27 20:06:11.067997 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.067846    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcdtt\" (UniqueName: \"kubernetes.io/projected/cc0ec0c9-8d78-4727-938c-582c7df21a42-kube-api-access-lcdtt\") pod \"cc0ec0c9-8d78-4727-938c-582c7df21a42\" (UID: \"cc0ec0c9-8d78-4727-938c-582c7df21a42\") "
Jul 27 20:06:11.068208 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.068182    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f37476f-34f8-421f-b16c-4d38261bd3fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f37476f-34f8-421f-b16c-4d38261bd3fd" (UID: "3f37476f-34f8-421f-b16c-4d38261bd3fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:11.068300 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.068278    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0ec0c9-8d78-4727-938c-582c7df21a42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc0ec0c9-8d78-4727-938c-582c7df21a42" (UID: "cc0ec0c9-8d78-4727-938c-582c7df21a42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:11.069950 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.069930    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f37476f-34f8-421f-b16c-4d38261bd3fd-kube-api-access-b7zzr" (OuterVolumeSpecName: "kube-api-access-b7zzr") pod "3f37476f-34f8-421f-b16c-4d38261bd3fd" (UID: "3f37476f-34f8-421f-b16c-4d38261bd3fd"). InnerVolumeSpecName "kube-api-access-b7zzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:11.070031 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.069971    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0ec0c9-8d78-4727-938c-582c7df21a42-kube-api-access-lcdtt" (OuterVolumeSpecName: "kube-api-access-lcdtt") pod "cc0ec0c9-8d78-4727-938c-582c7df21a42" (UID: "cc0ec0c9-8d78-4727-938c-582c7df21a42"). InnerVolumeSpecName "kube-api-access-lcdtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:11.169299 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.169232    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0ec0c9-8d78-4727-938c-582c7df21a42-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:11.169299 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.169293    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcdtt\" (UniqueName: \"kubernetes.io/projected/cc0ec0c9-8d78-4727-938c-582c7df21a42-kube-api-access-lcdtt\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:11.169299 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.169308    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7zzr\" (UniqueName: \"kubernetes.io/projected/3f37476f-34f8-421f-b16c-4d38261bd3fd-kube-api-access-b7zzr\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:11.169553 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.169319    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f37476f-34f8-421f-b16c-4d38261bd3fd-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:11.978321 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.978294    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"
Jul 27 20:06:11.978755 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.978294    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx" event={"ID":"3f37476f-34f8-421f-b16c-4d38261bd3fd","Type":"ContainerDied","Data":"fdc1a29f107089dc9308c13b9a691839b8c74e978f289fe65be4e6546f40ab58"}
Jul 27 20:06:11.978755 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.978461    2579 scope.go:117] "RemoveContainer" containerID="8ff689fca0639cb7d4032e7b61c6215596e2ffb738945f19007c396aa3a51eff"
Jul 27 20:06:11.979770 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.979607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s" event={"ID":"cc0ec0c9-8d78-4727-938c-582c7df21a42","Type":"ContainerDied","Data":"fb8d763656878fb3e8160e76eda394bc84218b65c5106ab1c0739e17fac93680"}
Jul 27 20:06:11.979770 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.979619    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"
Jul 27 20:06:11.987253 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:11.987207    2579 scope.go:117] "RemoveContainer" containerID="66ea0728675bf6435b99bd25e4afbaebc257f35653d5dd65b4d82e5d8d1d2686"
Jul 27 20:06:12.004033 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:12.004004    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"]
Jul 27 20:06:12.009502 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:12.009476    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-20b70e-54f64cd5cd-bh88s"]
Jul 27 20:06:12.019085 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:12.019045    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"]
Jul 27 20:06:12.024443 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:12.024414    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-20b70e-6b895885f6-twmgx"]
Jul 27 20:06:12.489587 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:12.489554    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f37476f-34f8-421f-b16c-4d38261bd3fd" path="/var/lib/kubelet/pods/3f37476f-34f8-421f-b16c-4d38261bd3fd/volumes"
Jul 27 20:06:12.489977 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:12.489963    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0ec0c9-8d78-4727-938c-582c7df21a42" path="/var/lib/kubelet/pods/cc0ec0c9-8d78-4727-938c-582c7df21a42/volumes"
Jul 27 20:06:14.291469 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.291427    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd" podUID="bafee261-bdae-485d-8edf-9506a788b468" containerName="main" containerID="cri-o://6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5" gracePeriod=45
Jul 27 20:06:14.536373 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.536346    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:06:14.601300 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601192    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjxzw\" (UniqueName: \"kubernetes.io/projected/bafee261-bdae-485d-8edf-9506a788b468-kube-api-access-kjxzw\") pod \"bafee261-bdae-485d-8edf-9506a788b468\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") "
Jul 27 20:06:14.601300 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601288    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-tmp-dir\") pod \"bafee261-bdae-485d-8edf-9506a788b468\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") "
Jul 27 20:06:14.601629 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601317    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-model-cache\") pod \"bafee261-bdae-485d-8edf-9506a788b468\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") "
Jul 27 20:06:14.601629 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601348    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-dshm\") pod \"bafee261-bdae-485d-8edf-9506a788b468\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") "
Jul 27 20:06:14.601629 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601389    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-home\") pod \"bafee261-bdae-485d-8edf-9506a788b468\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") "
Jul 27 20:06:14.601629 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601434    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bafee261-bdae-485d-8edf-9506a788b468-tls-certs\") pod \"bafee261-bdae-485d-8edf-9506a788b468\" (UID: \"bafee261-bdae-485d-8edf-9506a788b468\") "
Jul 27 20:06:14.601629 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601549    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-model-cache" (OuterVolumeSpecName: "model-cache") pod "bafee261-bdae-485d-8edf-9506a788b468" (UID: "bafee261-bdae-485d-8edf-9506a788b468"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:14.601629 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601588    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bafee261-bdae-485d-8edf-9506a788b468" (UID: "bafee261-bdae-485d-8edf-9506a788b468"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:14.601930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601673    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-home" (OuterVolumeSpecName: "home") pod "bafee261-bdae-485d-8edf-9506a788b468" (UID: "bafee261-bdae-485d-8edf-9506a788b468"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:14.601930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601829    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-home\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:14.601930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601845    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-tmp-dir\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:14.601930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.601853    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-model-cache\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:14.603479 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.603453    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bafee261-bdae-485d-8edf-9506a788b468-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bafee261-bdae-485d-8edf-9506a788b468" (UID: "bafee261-bdae-485d-8edf-9506a788b468"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:06:14.603594 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.603496    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-dshm" (OuterVolumeSpecName: "dshm") pod "bafee261-bdae-485d-8edf-9506a788b468" (UID: "bafee261-bdae-485d-8edf-9506a788b468"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:14.603594 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.603546    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bafee261-bdae-485d-8edf-9506a788b468-kube-api-access-kjxzw" (OuterVolumeSpecName: "kube-api-access-kjxzw") pod "bafee261-bdae-485d-8edf-9506a788b468" (UID: "bafee261-bdae-485d-8edf-9506a788b468"). InnerVolumeSpecName "kube-api-access-kjxzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:14.702336 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.702299    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bafee261-bdae-485d-8edf-9506a788b468-dshm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:14.702336 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.702332    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bafee261-bdae-485d-8edf-9506a788b468-tls-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:14.702336 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.702347    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjxzw\" (UniqueName: \"kubernetes.io/projected/bafee261-bdae-485d-8edf-9506a788b468-kube-api-access-kjxzw\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:14.998824 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.998787    2579 generic.go:356] "Generic (PLEG): container finished" podID="bafee261-bdae-485d-8edf-9506a788b468" containerID="6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5" exitCode=0
Jul 27 20:06:14.999015 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.998824    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd" event={"ID":"bafee261-bdae-485d-8edf-9506a788b468","Type":"ContainerDied","Data":"6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5"}
Jul 27 20:06:14.999015 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.998853    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"
Jul 27 20:06:14.999015 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.998876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd" event={"ID":"bafee261-bdae-485d-8edf-9506a788b468","Type":"ContainerDied","Data":"fcf23341169c9d639358d45c6d158ed213b938c8cf004b03390e8a22fda5e036"}
Jul 27 20:06:14.999015 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:14.998898    2579 scope.go:117] "RemoveContainer" containerID="6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5"
Jul 27 20:06:15.008516 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:15.008495    2579 scope.go:117] "RemoveContainer" containerID="6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5"
Jul 27 20:06:15.008796 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:06:15.008778    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5\": container with ID starting with 6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5 not found: ID does not exist" containerID="6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5"
Jul 27 20:06:15.008860 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:15.008806    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5"} err="failed to get container status \"6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5\": rpc error: code = NotFound desc = could not find container \"6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5\": container with ID starting with 6d05e6ba95779c14d915d3da0701f740734456f90088359ad05b5e0bf4e8b1d5 not found: ID does not exist"
Jul 27 20:06:15.020944 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:15.020911    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"]
Jul 27 20:06:15.023520 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:15.023494    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-20b70e/test-model-20b70e-kserve-754d5c8965-gdmhd"]
Jul 27 20:06:15.338467 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:15.338388    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"]
Jul 27 20:06:15.340620 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:15.340596    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20b70e-qkwlh"]
Jul 27 20:06:15.342681 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:15.342658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:15.722706 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:15.722677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:16.488900 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:16.488865    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39510c1c-89f7-4938-b72b-f9c5ab3be12b" path="/var/lib/kubelet/pods/39510c1c-89f7-4938-b72b-f9c5ab3be12b/volumes"
Jul 27 20:06:16.489320 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:16.489304    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bafee261-bdae-485d-8edf-9506a788b468" path="/var/lib/kubelet/pods/bafee261-bdae-485d-8edf-9506a788b468/volumes"
Jul 27 20:06:16.947469 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:16.947438    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:21.533862 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:21.533826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:21.636763 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:21.636725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:21.941606 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:21.941573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:22.144189 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144154    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"]
Jul 27 20:06:22.144833 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144813    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bafee261-bdae-485d-8edf-9506a788b468" containerName="main"
Jul 27 20:06:22.144884 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144838    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bafee261-bdae-485d-8edf-9506a788b468" containerName="main"
Jul 27 20:06:22.144884 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144850    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39510c1c-89f7-4938-b72b-f9c5ab3be12b" containerName="revoke-keys"
Jul 27 20:06:22.144884 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144858    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="39510c1c-89f7-4938-b72b-f9c5ab3be12b" containerName="revoke-keys"
Jul 27 20:06:22.144884 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144877    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0ec0c9-8d78-4727-938c-582c7df21a42" containerName="payload-processing"
Jul 27 20:06:22.145008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144886    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0ec0c9-8d78-4727-938c-582c7df21a42" containerName="payload-processing"
Jul 27 20:06:22.145008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144905    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f37476f-34f8-421f-b16c-4d38261bd3fd" containerName="payload-pre-processing"
Jul 27 20:06:22.145008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.144915    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f37476f-34f8-421f-b16c-4d38261bd3fd" containerName="payload-pre-processing"
Jul 27 20:06:22.145099 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.145022    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="39510c1c-89f7-4938-b72b-f9c5ab3be12b" containerName="revoke-keys"
Jul 27 20:06:22.145099 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.145033    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f37476f-34f8-421f-b16c-4d38261bd3fd" containerName="payload-pre-processing"
Jul 27 20:06:22.145099 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.145042    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bafee261-bdae-485d-8edf-9506a788b468" containerName="main"
Jul 27 20:06:22.145099 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.145050    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0ec0c9-8d78-4727-938c-582c7df21a42" containerName="payload-processing"
Jul 27 20:06:22.149965 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.149937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"
Jul 27 20:06:22.152825 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.152801    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 20:06:22.153433 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.153408    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"]
Jul 27 20:06:22.155357 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.155295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:22.278319 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.278222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24820bf1-ee6a-4b82-a078-b2772acc638b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-461c95-jpn87\" (UID: \"24820bf1-ee6a-4b82-a078-b2772acc638b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"
Jul 27 20:06:22.379467 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.379429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24820bf1-ee6a-4b82-a078-b2772acc638b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-461c95-jpn87\" (UID: \"24820bf1-ee6a-4b82-a078-b2772acc638b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"
Jul 27 20:06:22.380091 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.380070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24820bf1-ee6a-4b82-a078-b2772acc638b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-461c95-jpn87\" (UID: \"24820bf1-ee6a-4b82-a078-b2772acc638b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"
Jul 27 20:06:22.463425 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.463389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"
Jul 27 20:06:22.595849 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:22.595820    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"]
Jul 27 20:06:22.597906 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:06:22.597870    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24820bf1_ee6a_4b82_a078_b2772acc638b.slice/crio-03884411ecf7404870a49c8c3268d0ac04a5efeef4a2166fca81d55a886eb1f3 WatchSource:0}: Error finding container 03884411ecf7404870a49c8c3268d0ac04a5efeef4a2166fca81d55a886eb1f3: Status 404 returned error can't find the container with id 03884411ecf7404870a49c8c3268d0ac04a5efeef4a2166fca81d55a886eb1f3
Jul 27 20:06:23.042532 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:23.042494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87" event={"ID":"24820bf1-ee6a-4b82-a078-b2772acc638b","Type":"ContainerStarted","Data":"167509026d7f4d9568cae8f62b1e21cf8cf7961cfc8993aed2fcda7ea802e063"}
Jul 27 20:06:23.042696 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:23.042551    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87" event={"ID":"24820bf1-ee6a-4b82-a078-b2772acc638b","Type":"ContainerStarted","Data":"03884411ecf7404870a49c8c3268d0ac04a5efeef4a2166fca81d55a886eb1f3"}
Jul 27 20:06:23.058697 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:23.058647    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87" podStartSLOduration=1.058632789 podStartE2EDuration="1.058632789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:06:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:06:23.056657515 +0000 UTC m=+2435.148009014" watchObservedRunningTime="2026-07-27 20:06:23.058632789 +0000 UTC m=+2435.149984286"
Jul 27 20:06:24.049950 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:24.049913    2579 generic.go:356] "Generic (PLEG): container finished" podID="24820bf1-ee6a-4b82-a078-b2772acc638b" containerID="167509026d7f4d9568cae8f62b1e21cf8cf7961cfc8993aed2fcda7ea802e063" exitCode=0
Jul 27 20:06:24.050369 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:24.049999    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87" event={"ID":"24820bf1-ee6a-4b82-a078-b2772acc638b","Type":"ContainerDied","Data":"167509026d7f4d9568cae8f62b1e21cf8cf7961cfc8993aed2fcda7ea802e063"}
Jul 27 20:06:25.183719 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:25.183695    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"
Jul 27 20:06:25.307665 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:25.307559    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24820bf1-ee6a-4b82-a078-b2772acc638b-maas-api-service-ca\") pod \"24820bf1-ee6a-4b82-a078-b2772acc638b\" (UID: \"24820bf1-ee6a-4b82-a078-b2772acc638b\") "
Jul 27 20:06:25.307943 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:25.307921    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24820bf1-ee6a-4b82-a078-b2772acc638b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "24820bf1-ee6a-4b82-a078-b2772acc638b" (UID: "24820bf1-ee6a-4b82-a078-b2772acc638b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:25.409263 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:25.409204    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24820bf1-ee6a-4b82-a078-b2772acc638b-maas-api-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:26.062149 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:26.062115    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87" event={"ID":"24820bf1-ee6a-4b82-a078-b2772acc638b","Type":"ContainerDied","Data":"03884411ecf7404870a49c8c3268d0ac04a5efeef4a2166fca81d55a886eb1f3"}
Jul 27 20:06:26.062346 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:26.062168    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03884411ecf7404870a49c8c3268d0ac04a5efeef4a2166fca81d55a886eb1f3"
Jul 27 20:06:26.062346 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:26.062123    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"
Jul 27 20:06:27.482754 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:27.482720    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:32.586443 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.586402    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"]
Jul 27 20:06:32.587044 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.586669    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b" podUID="fed289c2-3d82-4cc9-b039-a29e77f9b1b3" containerName="payload-pre-processing" containerID="cri-o://0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32" gracePeriod=30
Jul 27 20:06:32.646363 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.646327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:32.747972 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.747933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:32.850136 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.850076    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:06:32.977645 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.977615    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mzjz\" (UniqueName: \"kubernetes.io/projected/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-kube-api-access-8mzjz\") pod \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\" (UID: \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\") "
Jul 27 20:06:32.977834 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.977676    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-plugins-config-volume\") pod \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\" (UID: \"fed289c2-3d82-4cc9-b039-a29e77f9b1b3\") "
Jul 27 20:06:32.978057 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.978029    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fed289c2-3d82-4cc9-b039-a29e77f9b1b3" (UID: "fed289c2-3d82-4cc9-b039-a29e77f9b1b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:32.979580 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:32.979549    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-kube-api-access-8mzjz" (OuterVolumeSpecName: "kube-api-access-8mzjz") pod "fed289c2-3d82-4cc9-b039-a29e77f9b1b3" (UID: "fed289c2-3d82-4cc9-b039-a29e77f9b1b3"). InnerVolumeSpecName "kube-api-access-8mzjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:33.079136 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.079103    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mzjz\" (UniqueName: \"kubernetes.io/projected/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-kube-api-access-8mzjz\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:33.079136 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.079132    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed289c2-3d82-4cc9-b039-a29e77f9b1b3-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:06:33.097400 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.097371    2579 generic.go:356] "Generic (PLEG): container finished" podID="fed289c2-3d82-4cc9-b039-a29e77f9b1b3" containerID="0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32" exitCode=0
Jul 27 20:06:33.097545 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.097415    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"
Jul 27 20:06:33.097545 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.097423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b" event={"ID":"fed289c2-3d82-4cc9-b039-a29e77f9b1b3","Type":"ContainerDied","Data":"0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32"}
Jul 27 20:06:33.097545 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.097466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b" event={"ID":"fed289c2-3d82-4cc9-b039-a29e77f9b1b3","Type":"ContainerDied","Data":"060cbfea2633f55c8e4c5adf1d5f083c334e8dd588c8b969f713e7a7946959c4"}
Jul 27 20:06:33.097545 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.097490    2579 scope.go:117] "RemoveContainer" containerID="0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32"
Jul 27 20:06:33.106658 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.106639    2579 scope.go:117] "RemoveContainer" containerID="0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32"
Jul 27 20:06:33.106938 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:06:33.106915    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32\": container with ID starting with 0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32 not found: ID does not exist" containerID="0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32"
Jul 27 20:06:33.107021 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.106950    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32"} err="failed to get container status \"0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32\": rpc error: code = NotFound desc = could not find container \"0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32\": container with ID starting with 0c88c1d87037cb87bcd7bd249be33ecd03c22b1bf1e0b37b31a83e29f7639d32 not found: ID does not exist"
Jul 27 20:06:33.118604 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.118579    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"]
Jul 27 20:06:33.122090 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:33.122063    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-461c95-5c7459f788-xp96b"]
Jul 27 20:06:34.488631 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:34.488596    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed289c2-3d82-4cc9-b039-a29e77f9b1b3" path="/var/lib/kubelet/pods/fed289c2-3d82-4cc9-b039-a29e77f9b1b3/volumes"
Jul 27 20:06:37.212606 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:37.212567    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"]
Jul 27 20:06:37.214861 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:37.214839    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-461c95-jpn87"]
Jul 27 20:06:37.225439 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:37.225417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:37.626161 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:37.626050    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:38.489272 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:38.489208    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24820bf1-ee6a-4b82-a078-b2772acc638b" path="/var/lib/kubelet/pods/24820bf1-ee6a-4b82-a078-b2772acc638b/volumes"
Jul 27 20:06:43.497296 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:43.497261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:43.570802 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:43.570764    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:43.758523 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:43.758420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:43.960412 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:43.960372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:44.300918 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:44.300883    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:44.598788 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:44.598680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:45.005268 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:45.005223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:47.857760 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:47.857725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:48.023683 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.023644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:48.416444 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.416404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:48.513933 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.513902    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"]
Jul 27 20:06:48.514335 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.514321    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24820bf1-ee6a-4b82-a078-b2772acc638b" containerName="revoke-keys"
Jul 27 20:06:48.514335 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.514336    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="24820bf1-ee6a-4b82-a078-b2772acc638b" containerName="revoke-keys"
Jul 27 20:06:48.514418 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.514353    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed289c2-3d82-4cc9-b039-a29e77f9b1b3" containerName="payload-pre-processing"
Jul 27 20:06:48.514418 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.514358    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed289c2-3d82-4cc9-b039-a29e77f9b1b3" containerName="payload-pre-processing"
Jul 27 20:06:48.514418 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.514414    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed289c2-3d82-4cc9-b039-a29e77f9b1b3" containerName="payload-pre-processing"
Jul 27 20:06:48.514535 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.514424    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="24820bf1-ee6a-4b82-a078-b2772acc638b" containerName="revoke-keys"
Jul 27 20:06:48.518917 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.518899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:48.521844 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.521823    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-110a90-dockercfg-q25w8\""
Jul 27 20:06:48.521844 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.521835    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-110a90\""
Jul 27 20:06:48.525367 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.525333    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a487cc13-4358-492a-a112-0fe481195376-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7\" (UID: \"a487cc13-4358-492a-a112-0fe481195376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:48.525496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.525392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fr75\" (UniqueName: \"kubernetes.io/projected/a487cc13-4358-492a-a112-0fe481195376-kube-api-access-5fr75\") pod \"payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7\" (UID: \"a487cc13-4358-492a-a112-0fe481195376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:48.528673 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.528637    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:48.529400 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.529378    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"]
Jul 27 20:06:48.558834 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.556879    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"]
Jul 27 20:06:48.562554 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.562532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:48.569189 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.569163    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"]
Jul 27 20:06:48.626491 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.626462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a487cc13-4358-492a-a112-0fe481195376-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7\" (UID: \"a487cc13-4358-492a-a112-0fe481195376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:48.626491 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.626496    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-plugins-config-volume\") pod \"payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm\" (UID: \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\") " pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:48.626754 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.626562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fr75\" (UniqueName: \"kubernetes.io/projected/a487cc13-4358-492a-a112-0fe481195376-kube-api-access-5fr75\") pod \"payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7\" (UID: \"a487cc13-4358-492a-a112-0fe481195376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:48.626754 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.626607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br697\" (UniqueName: \"kubernetes.io/projected/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-kube-api-access-br697\") pod \"payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm\" (UID: \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\") " pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:48.627402 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.627377    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a487cc13-4358-492a-a112-0fe481195376-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7\" (UID: \"a487cc13-4358-492a-a112-0fe481195376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:48.634642 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.634613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fr75\" (UniqueName: \"kubernetes.io/projected/a487cc13-4358-492a-a112-0fe481195376-kube-api-access-5fr75\") pod \"payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7\" (UID: \"a487cc13-4358-492a-a112-0fe481195376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:48.727159 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.727122    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-plugins-config-volume\") pod \"payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm\" (UID: \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\") " pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:48.727378 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.727201    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br697\" (UniqueName: \"kubernetes.io/projected/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-kube-api-access-br697\") pod \"payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm\" (UID: \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\") " pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:48.727843 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.727822    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-plugins-config-volume\") pod \"payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm\" (UID: \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\") " pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:48.734584 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.734558    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br697\" (UniqueName: \"kubernetes.io/projected/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-kube-api-access-br697\") pod \"payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm\" (UID: \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\") " pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:48.831074 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.831033    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:48.875190 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.875158    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:48.989260 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:48.989211    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"]
Jul 27 20:06:48.990959 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:06:48.990929    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda487cc13_4358_492a_a112_0fe481195376.slice/crio-71c4bf1f0e7d6247667ad9c5d47b5ae1e9c86f84c6fc9171464316c0cf92fcc7 WatchSource:0}: Error finding container 71c4bf1f0e7d6247667ad9c5d47b5ae1e9c86f84c6fc9171464316c0cf92fcc7: Status 404 returned error can't find the container with id 71c4bf1f0e7d6247667ad9c5d47b5ae1e9c86f84c6fc9171464316c0cf92fcc7
Jul 27 20:06:49.033057 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.033032    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"]
Jul 27 20:06:49.035066 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:06:49.035041    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ec7652_8a1b_45cf_90f2_10f8eca7a200.slice/crio-cab985568a0ddd437efdc50b334f17d1262d40136ba173e1e0a388cebab194e4 WatchSource:0}: Error finding container cab985568a0ddd437efdc50b334f17d1262d40136ba173e1e0a388cebab194e4: Status 404 returned error can't find the container with id cab985568a0ddd437efdc50b334f17d1262d40136ba173e1e0a388cebab194e4
Jul 27 20:06:49.191792 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.191753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm" event={"ID":"f1ec7652-8a1b-45cf-90f2-10f8eca7a200","Type":"ContainerStarted","Data":"3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4"}
Jul 27 20:06:49.191969 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.191812    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm" event={"ID":"f1ec7652-8a1b-45cf-90f2-10f8eca7a200","Type":"ContainerStarted","Data":"cab985568a0ddd437efdc50b334f17d1262d40136ba173e1e0a388cebab194e4"}
Jul 27 20:06:49.191969 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.191843    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:06:49.193286 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.193221    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7" event={"ID":"a487cc13-4358-492a-a112-0fe481195376","Type":"ContainerStarted","Data":"2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613"}
Jul 27 20:06:49.193401 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.193298    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7" event={"ID":"a487cc13-4358-492a-a112-0fe481195376","Type":"ContainerStarted","Data":"71c4bf1f0e7d6247667ad9c5d47b5ae1e9c86f84c6fc9171464316c0cf92fcc7"}
Jul 27 20:06:49.193401 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.193329    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:06:49.210216 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.210164    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm" podStartSLOduration=1.21013335 podStartE2EDuration="1.21013335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:06:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:06:49.208052707 +0000 UTC m=+2461.299404205" watchObservedRunningTime="2026-07-27 20:06:49.21013335 +0000 UTC m=+2461.301484842"
Jul 27 20:06:49.227876 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:49.227829    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7" podStartSLOduration=1.227811343 podStartE2EDuration="1.227811343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:06:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:06:49.226848455 +0000 UTC m=+2461.318199977" watchObservedRunningTime="2026-07-27 20:06:49.227811343 +0000 UTC m=+2461.319162885"
Jul 27 20:06:52.388452 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:52.388421    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:52.807230 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:52.807197    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:53.097625 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:53.097535    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:53.346575 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:53.346540    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:55.862682 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:55.862652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:55.976119 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:55.976084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:56.070492 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.070454    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:56.449203 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.449176    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:56.543280 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.543014    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"]
Jul 27 20:06:56.547344 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.547310    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:06:56.548400 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.548379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:56.550614 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.550592    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-16dc17\""
Jul 27 20:06:56.550751 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.550673    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-16dc17-dockercfg-42ll4\""
Jul 27 20:06:56.560431 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.560409    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"]
Jul 27 20:06:56.610629 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.610586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmt85\" (UniqueName: \"kubernetes.io/projected/373430b2-86b7-41d7-a059-2781d17e00d9-kube-api-access-hmt85\") pod \"payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj\" (UID: \"373430b2-86b7-41d7-a059-2781d17e00d9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:56.610815 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.610651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373430b2-86b7-41d7-a059-2781d17e00d9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj\" (UID: \"373430b2-86b7-41d7-a059-2781d17e00d9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:56.711222 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.711139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmt85\" (UniqueName: \"kubernetes.io/projected/373430b2-86b7-41d7-a059-2781d17e00d9-kube-api-access-hmt85\") pod \"payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj\" (UID: \"373430b2-86b7-41d7-a059-2781d17e00d9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:56.711222 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.711202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373430b2-86b7-41d7-a059-2781d17e00d9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj\" (UID: \"373430b2-86b7-41d7-a059-2781d17e00d9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:56.712046 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.712022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373430b2-86b7-41d7-a059-2781d17e00d9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj\" (UID: \"373430b2-86b7-41d7-a059-2781d17e00d9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:56.719059 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.719032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmt85\" (UniqueName: \"kubernetes.io/projected/373430b2-86b7-41d7-a059-2781d17e00d9-kube-api-access-hmt85\") pod \"payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj\" (UID: \"373430b2-86b7-41d7-a059-2781d17e00d9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:56.859847 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.859804    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:56.993838 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:56.993812    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"]
Jul 27 20:06:56.995686 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:06:56.995656    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373430b2_86b7_41d7_a059_2781d17e00d9.slice/crio-1a5750025e6a9d071f97b5011ac27f1551e6539441a0b09fbb506c06d5531dc0 WatchSource:0}: Error finding container 1a5750025e6a9d071f97b5011ac27f1551e6539441a0b09fbb506c06d5531dc0: Status 404 returned error can't find the container with id 1a5750025e6a9d071f97b5011ac27f1551e6539441a0b09fbb506c06d5531dc0
Jul 27 20:06:57.235122 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:57.235071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj" event={"ID":"373430b2-86b7-41d7-a059-2781d17e00d9","Type":"ContainerStarted","Data":"624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9"}
Jul 27 20:06:57.235346 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:57.235152    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:06:57.235346 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:57.235179    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj" event={"ID":"373430b2-86b7-41d7-a059-2781d17e00d9","Type":"ContainerStarted","Data":"1a5750025e6a9d071f97b5011ac27f1551e6539441a0b09fbb506c06d5531dc0"}
Jul 27 20:06:57.251607 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:06:57.251514    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj" podStartSLOduration=1.2514971990000001 podStartE2EDuration="1.251497199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:06:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:06:57.250590902 +0000 UTC m=+2469.341942402" watchObservedRunningTime="2026-07-27 20:06:57.251497199 +0000 UTC m=+2469.342848704"
Jul 27 20:07:00.200324 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:00.200295    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:07:00.200724 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:00.200377    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:07:00.358072 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:00.358039    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:08.243006 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:08.242977    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:07:23.950900 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:23.950851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:23.971141 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:23.970955    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"]
Jul 27 20:07:23.971359 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:23.971216    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm" podUID="f1ec7652-8a1b-45cf-90f2-10f8eca7a200" containerName="payload-processing" containerID="cri-o://3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4" gracePeriod=30
Jul 27 20:07:23.979002 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:23.978965    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"]
Jul 27 20:07:23.979445 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:23.979382    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7" podUID="a487cc13-4358-492a-a112-0fe481195376" containerName="payload-pre-processing" containerID="cri-o://2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613" gracePeriod=30
Jul 27 20:07:24.051352 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.051160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:24.116294 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.116227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:24.246226 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.246199    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:07:24.249255 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.249221    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:07:24.284045 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.284016    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a487cc13-4358-492a-a112-0fe481195376-plugins-config-volume\") pod \"a487cc13-4358-492a-a112-0fe481195376\" (UID: \"a487cc13-4358-492a-a112-0fe481195376\") "
Jul 27 20:07:24.284191 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.284080    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fr75\" (UniqueName: \"kubernetes.io/projected/a487cc13-4358-492a-a112-0fe481195376-kube-api-access-5fr75\") pod \"a487cc13-4358-492a-a112-0fe481195376\" (UID: \"a487cc13-4358-492a-a112-0fe481195376\") "
Jul 27 20:07:24.284273 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.284192    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-plugins-config-volume\") pod \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\" (UID: \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\") "
Jul 27 20:07:24.284273 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.284255    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-br697\" (UniqueName: \"kubernetes.io/projected/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-kube-api-access-br697\") pod \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\" (UID: \"f1ec7652-8a1b-45cf-90f2-10f8eca7a200\") "
Jul 27 20:07:24.284552 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.284521    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a487cc13-4358-492a-a112-0fe481195376-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a487cc13-4358-492a-a112-0fe481195376" (UID: "a487cc13-4358-492a-a112-0fe481195376"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:07:24.284681 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.284570    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1ec7652-8a1b-45cf-90f2-10f8eca7a200" (UID: "f1ec7652-8a1b-45cf-90f2-10f8eca7a200"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:07:24.284681 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.284619    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a487cc13-4358-492a-a112-0fe481195376-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:07:24.286358 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.286333    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-kube-api-access-br697" (OuterVolumeSpecName: "kube-api-access-br697") pod "f1ec7652-8a1b-45cf-90f2-10f8eca7a200" (UID: "f1ec7652-8a1b-45cf-90f2-10f8eca7a200"). InnerVolumeSpecName "kube-api-access-br697". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:07:24.286465 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.286447    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a487cc13-4358-492a-a112-0fe481195376-kube-api-access-5fr75" (OuterVolumeSpecName: "kube-api-access-5fr75") pod "a487cc13-4358-492a-a112-0fe481195376" (UID: "a487cc13-4358-492a-a112-0fe481195376"). InnerVolumeSpecName "kube-api-access-5fr75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:07:24.385172 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.385138    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fr75\" (UniqueName: \"kubernetes.io/projected/a487cc13-4358-492a-a112-0fe481195376-kube-api-access-5fr75\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:07:24.385172 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.385169    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:07:24.385404 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.385184    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br697\" (UniqueName: \"kubernetes.io/projected/f1ec7652-8a1b-45cf-90f2-10f8eca7a200-kube-api-access-br697\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:07:24.385404 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.385343    2579 generic.go:356] "Generic (PLEG): container finished" podID="f1ec7652-8a1b-45cf-90f2-10f8eca7a200" containerID="3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4" exitCode=0
Jul 27 20:07:24.385404 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.385387    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"
Jul 27 20:07:24.385552 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.385428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm" event={"ID":"f1ec7652-8a1b-45cf-90f2-10f8eca7a200","Type":"ContainerDied","Data":"3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4"}
Jul 27 20:07:24.385552 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.385480    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm" event={"ID":"f1ec7652-8a1b-45cf-90f2-10f8eca7a200","Type":"ContainerDied","Data":"cab985568a0ddd437efdc50b334f17d1262d40136ba173e1e0a388cebab194e4"}
Jul 27 20:07:24.385552 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.385507    2579 scope.go:117] "RemoveContainer" containerID="3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4"
Jul 27 20:07:24.386818 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.386795    2579 generic.go:356] "Generic (PLEG): container finished" podID="a487cc13-4358-492a-a112-0fe481195376" containerID="2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613" exitCode=0
Jul 27 20:07:24.386929 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.386834    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"
Jul 27 20:07:24.386929 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.386881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7" event={"ID":"a487cc13-4358-492a-a112-0fe481195376","Type":"ContainerDied","Data":"2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613"}
Jul 27 20:07:24.386929 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.386921    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7" event={"ID":"a487cc13-4358-492a-a112-0fe481195376","Type":"ContainerDied","Data":"71c4bf1f0e7d6247667ad9c5d47b5ae1e9c86f84c6fc9171464316c0cf92fcc7"}
Jul 27 20:07:24.395906 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.395890    2579 scope.go:117] "RemoveContainer" containerID="3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4"
Jul 27 20:07:24.396147 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:07:24.396120    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4\": container with ID starting with 3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4 not found: ID does not exist" containerID="3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4"
Jul 27 20:07:24.396216 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.396159    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4"} err="failed to get container status \"3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4\": rpc error: code = NotFound desc = could not find container \"3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4\": container with ID starting with 3047cd6fb3fa0cba5257ffcaa673a6e61143b30eb086065e4209f2c9eec191c4 not found: ID does not exist"
Jul 27 20:07:24.396216 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.396183    2579 scope.go:117] "RemoveContainer" containerID="2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613"
Jul 27 20:07:24.404134 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.404121    2579 scope.go:117] "RemoveContainer" containerID="2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613"
Jul 27 20:07:24.404384 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:07:24.404364    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613\": container with ID starting with 2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613 not found: ID does not exist" containerID="2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613"
Jul 27 20:07:24.404457 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.404394    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613"} err="failed to get container status \"2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613\": rpc error: code = NotFound desc = could not find container \"2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613\": container with ID starting with 2ba0616d2dac28ac2f444ba41db44a9aab70e9ab177e748cec402e660f71e613 not found: ID does not exist"
Jul 27 20:07:24.409837 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.409818    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"]
Jul 27 20:07:24.413406 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.413388    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-110a90-856df9b96-59ps7"]
Jul 27 20:07:24.422944 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.422924    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"]
Jul 27 20:07:24.424595 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.424576    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-110a90-6476bd6ccb-c6xlm"]
Jul 27 20:07:24.489901 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.489823    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a487cc13-4358-492a-a112-0fe481195376" path="/var/lib/kubelet/pods/a487cc13-4358-492a-a112-0fe481195376/volumes"
Jul 27 20:07:24.490193 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:24.490180    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ec7652-8a1b-45cf-90f2-10f8eca7a200" path="/var/lib/kubelet/pods/f1ec7652-8a1b-45cf-90f2-10f8eca7a200/volumes"
Jul 27 20:07:28.415134 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:28.415100    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:28.803879 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:28.803845    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:34.748127 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:34.748084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:34.848740 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:34.848708    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:35.014923 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.014822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:35.226498 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.226470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:35.342166 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.342082    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"]
Jul 27 20:07:35.342742 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.342719    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ec7652-8a1b-45cf-90f2-10f8eca7a200" containerName="payload-processing"
Jul 27 20:07:35.342742 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.342742    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ec7652-8a1b-45cf-90f2-10f8eca7a200" containerName="payload-processing"
Jul 27 20:07:35.342937 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.342757    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a487cc13-4358-492a-a112-0fe481195376" containerName="payload-pre-processing"
Jul 27 20:07:35.342937 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.342765    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a487cc13-4358-492a-a112-0fe481195376" containerName="payload-pre-processing"
Jul 27 20:07:35.342937 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.342919    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a487cc13-4358-492a-a112-0fe481195376" containerName="payload-pre-processing"
Jul 27 20:07:35.342937 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.342932    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ec7652-8a1b-45cf-90f2-10f8eca7a200" containerName="payload-processing"
Jul 27 20:07:35.347972 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.347947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"
Jul 27 20:07:35.350463 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.350438    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 20:07:35.352029 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.351987    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"]
Jul 27 20:07:35.381721 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.381681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d460ae61-72cd-427e-b27c-1a70275c92e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx\" (UID: \"d460ae61-72cd-427e-b27c-1a70275c92e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"
Jul 27 20:07:35.482491 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.482452    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d460ae61-72cd-427e-b27c-1a70275c92e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx\" (UID: \"d460ae61-72cd-427e-b27c-1a70275c92e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"
Jul 27 20:07:35.483106 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.483087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d460ae61-72cd-427e-b27c-1a70275c92e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx\" (UID: \"d460ae61-72cd-427e-b27c-1a70275c92e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"
Jul 27 20:07:35.661926 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.661844    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"
Jul 27 20:07:35.794885 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:35.794859    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"]
Jul 27 20:07:35.796984 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:07:35.796959    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd460ae61_72cd_427e_b27c_1a70275c92e4.slice/crio-23eae253bd516e42cc5e0d7554f7e589c691c5a35ff7ed6467f3be96dca44797 WatchSource:0}: Error finding container 23eae253bd516e42cc5e0d7554f7e589c691c5a35ff7ed6467f3be96dca44797: Status 404 returned error can't find the container with id 23eae253bd516e42cc5e0d7554f7e589c691c5a35ff7ed6467f3be96dca44797
Jul 27 20:07:36.458341 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:36.458307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx" event={"ID":"d460ae61-72cd-427e-b27c-1a70275c92e4","Type":"ContainerStarted","Data":"cd32cc8cd0c4be6dc3a6e1495734c3ef1cc9ac9cfa1d7b96f974bdbd1628d5a2"}
Jul 27 20:07:36.458515 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:36.458360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx" event={"ID":"d460ae61-72cd-427e-b27c-1a70275c92e4","Type":"ContainerStarted","Data":"23eae253bd516e42cc5e0d7554f7e589c691c5a35ff7ed6467f3be96dca44797"}
Jul 27 20:07:36.474324 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:36.474227    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx" podStartSLOduration=1.4742122229999999 podStartE2EDuration="1.474212223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:07:36.472559409 +0000 UTC m=+2508.563910906" watchObservedRunningTime="2026-07-27 20:07:36.474212223 +0000 UTC m=+2508.565563720"
Jul 27 20:07:37.464759 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:37.464724    2579 generic.go:356] "Generic (PLEG): container finished" podID="d460ae61-72cd-427e-b27c-1a70275c92e4" containerID="cd32cc8cd0c4be6dc3a6e1495734c3ef1cc9ac9cfa1d7b96f974bdbd1628d5a2" exitCode=0
Jul 27 20:07:37.465160 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:37.464798    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx" event={"ID":"d460ae61-72cd-427e-b27c-1a70275c92e4","Type":"ContainerDied","Data":"cd32cc8cd0c4be6dc3a6e1495734c3ef1cc9ac9cfa1d7b96f974bdbd1628d5a2"}
Jul 27 20:07:38.603279 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:38.603253    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"
Jul 27 20:07:38.713205 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:38.713164    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d460ae61-72cd-427e-b27c-1a70275c92e4-maas-api-service-ca\") pod \"d460ae61-72cd-427e-b27c-1a70275c92e4\" (UID: \"d460ae61-72cd-427e-b27c-1a70275c92e4\") "
Jul 27 20:07:38.713610 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:38.713588    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d460ae61-72cd-427e-b27c-1a70275c92e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d460ae61-72cd-427e-b27c-1a70275c92e4" (UID: "d460ae61-72cd-427e-b27c-1a70275c92e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:07:38.814184 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:38.814099    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d460ae61-72cd-427e-b27c-1a70275c92e4-maas-api-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:07:39.477447 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:39.477415    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"
Jul 27 20:07:39.477447 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:39.477430    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx" event={"ID":"d460ae61-72cd-427e-b27c-1a70275c92e4","Type":"ContainerDied","Data":"23eae253bd516e42cc5e0d7554f7e589c691c5a35ff7ed6467f3be96dca44797"}
Jul 27 20:07:39.477750 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:39.477475    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23eae253bd516e42cc5e0d7554f7e589c691c5a35ff7ed6467f3be96dca44797"
Jul 27 20:07:50.990962 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:50.990928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:51.012825 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.012791    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"]
Jul 27 20:07:51.013317 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.013264    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj" podUID="373430b2-86b7-41d7-a059-2781d17e00d9" containerName="payload-pre-processing" containerID="cri-o://624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9" gracePeriod=30
Jul 27 20:07:51.090745 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.090712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:51.268472 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.268444    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:07:51.336532 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.336502    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373430b2-86b7-41d7-a059-2781d17e00d9-plugins-config-volume\") pod \"373430b2-86b7-41d7-a059-2781d17e00d9\" (UID: \"373430b2-86b7-41d7-a059-2781d17e00d9\") "
Jul 27 20:07:51.336708 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.336565    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmt85\" (UniqueName: \"kubernetes.io/projected/373430b2-86b7-41d7-a059-2781d17e00d9-kube-api-access-hmt85\") pod \"373430b2-86b7-41d7-a059-2781d17e00d9\" (UID: \"373430b2-86b7-41d7-a059-2781d17e00d9\") "
Jul 27 20:07:51.336880 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.336856    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/373430b2-86b7-41d7-a059-2781d17e00d9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "373430b2-86b7-41d7-a059-2781d17e00d9" (UID: "373430b2-86b7-41d7-a059-2781d17e00d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:07:51.338759 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.338723    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/373430b2-86b7-41d7-a059-2781d17e00d9-kube-api-access-hmt85" (OuterVolumeSpecName: "kube-api-access-hmt85") pod "373430b2-86b7-41d7-a059-2781d17e00d9" (UID: "373430b2-86b7-41d7-a059-2781d17e00d9"). InnerVolumeSpecName "kube-api-access-hmt85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:07:51.438115 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.438077    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmt85\" (UniqueName: \"kubernetes.io/projected/373430b2-86b7-41d7-a059-2781d17e00d9-kube-api-access-hmt85\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:07:51.438115 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.438110    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373430b2-86b7-41d7-a059-2781d17e00d9-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:07:51.542616 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.542528    2579 generic.go:356] "Generic (PLEG): container finished" podID="373430b2-86b7-41d7-a059-2781d17e00d9" containerID="624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9" exitCode=0
Jul 27 20:07:51.542616 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.542579    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"
Jul 27 20:07:51.542857 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.542617    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj" event={"ID":"373430b2-86b7-41d7-a059-2781d17e00d9","Type":"ContainerDied","Data":"624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9"}
Jul 27 20:07:51.542857 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.542674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj" event={"ID":"373430b2-86b7-41d7-a059-2781d17e00d9","Type":"ContainerDied","Data":"1a5750025e6a9d071f97b5011ac27f1551e6539441a0b09fbb506c06d5531dc0"}
Jul 27 20:07:51.542857 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.542697    2579 scope.go:117] "RemoveContainer" containerID="624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9"
Jul 27 20:07:51.552525 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.552503    2579 scope.go:117] "RemoveContainer" containerID="624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9"
Jul 27 20:07:51.552845 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:07:51.552825    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9\": container with ID starting with 624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9 not found: ID does not exist" containerID="624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9"
Jul 27 20:07:51.552921 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.552855    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9"} err="failed to get container status \"624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9\": rpc error: code = NotFound desc = could not find container \"624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9\": container with ID starting with 624d6bd32de973c85674996dc65d874eb9d095f54c0abb48a6dd36ba009bdbd9 not found: ID does not exist"
Jul 27 20:07:51.564844 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.564816    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"]
Jul 27 20:07:51.568204 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:51.568182    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-16dc17-6749d9d9dd-dpbkj"]
Jul 27 20:07:52.488864 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:52.488834    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="373430b2-86b7-41d7-a059-2781d17e00d9" path="/var/lib/kubelet/pods/373430b2-86b7-41d7-a059-2781d17e00d9/volumes"
Jul 27 20:07:55.421662 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:55.421610    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"]
Jul 27 20:07:55.423388 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:55.423368    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16dc17-dfxjx"]
Jul 27 20:07:55.526876 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:55.526840    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:55.822647 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:55.822614    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:07:56.489065 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:07:56.489036    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d460ae61-72cd-427e-b27c-1a70275c92e4" path="/var/lib/kubelet/pods/d460ae61-72cd-427e-b27c-1a70275c92e4/volumes"
Jul 27 20:08:01.480922 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:01.480886    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:01.582920 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:01.582891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:01.731117 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:01.731003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:01.991447 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:01.991367    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:02.334422 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:02.334339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:02.649199 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:02.649119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:02.775263 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:02.775212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:05.222822 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.222779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:05.335089 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.335051    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:05.447494 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.447460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:05.552051 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.551948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:05.851323 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.851214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:05.941393 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.941362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:05.942204 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.942178    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"]
Jul 27 20:08:05.942611 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.942597    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d460ae61-72cd-427e-b27c-1a70275c92e4" containerName="revoke-keys"
Jul 27 20:08:05.942611 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.942612    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d460ae61-72cd-427e-b27c-1a70275c92e4" containerName="revoke-keys"
Jul 27 20:08:05.942711 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.942625    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="373430b2-86b7-41d7-a059-2781d17e00d9" containerName="payload-pre-processing"
Jul 27 20:08:05.942711 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.942631    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="373430b2-86b7-41d7-a059-2781d17e00d9" containerName="payload-pre-processing"
Jul 27 20:08:05.942711 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.942690    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d460ae61-72cd-427e-b27c-1a70275c92e4" containerName="revoke-keys"
Jul 27 20:08:05.942711 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.942699    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="373430b2-86b7-41d7-a059-2781d17e00d9" containerName="payload-pre-processing"
Jul 27 20:08:05.945593 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.945577    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:05.947896 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.947877    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-97e28d-dockercfg-7kwc2\""
Jul 27 20:08:05.948031 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.947922    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-97e28d\""
Jul 27 20:08:05.954083 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.954058    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"]
Jul 27 20:08:05.972748 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.972718    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"]
Jul 27 20:08:05.976154 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.976138    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:05.985436 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:05.985408    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"]
Jul 27 20:08:06.075948 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.075912    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0150142d-7c7a-4413-acd3-b3b196107d58-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh\" (UID: \"0150142d-7c7a-4413-acd3-b3b196107d58\") " pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:06.076121 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.075966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n94pv\" (UniqueName: \"kubernetes.io/projected/0150142d-7c7a-4413-acd3-b3b196107d58-kube-api-access-n94pv\") pod \"payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh\" (UID: \"0150142d-7c7a-4413-acd3-b3b196107d58\") " pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:06.076121 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.075990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ff142c-a911-46bf-b471-7825d327add4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz\" (UID: \"13ff142c-a911-46bf-b471-7825d327add4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:06.076121 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.076062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8mrt\" (UniqueName: \"kubernetes.io/projected/13ff142c-a911-46bf-b471-7825d327add4-kube-api-access-g8mrt\") pod \"payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz\" (UID: \"13ff142c-a911-46bf-b471-7825d327add4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:06.176935 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.176839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8mrt\" (UniqueName: \"kubernetes.io/projected/13ff142c-a911-46bf-b471-7825d327add4-kube-api-access-g8mrt\") pod \"payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz\" (UID: \"13ff142c-a911-46bf-b471-7825d327add4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:06.176935 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.176884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0150142d-7c7a-4413-acd3-b3b196107d58-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh\" (UID: \"0150142d-7c7a-4413-acd3-b3b196107d58\") " pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:06.177170 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.176955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n94pv\" (UniqueName: \"kubernetes.io/projected/0150142d-7c7a-4413-acd3-b3b196107d58-kube-api-access-n94pv\") pod \"payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh\" (UID: \"0150142d-7c7a-4413-acd3-b3b196107d58\") " pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:06.177170 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.176993    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ff142c-a911-46bf-b471-7825d327add4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz\" (UID: \"13ff142c-a911-46bf-b471-7825d327add4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:06.177668 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.177648    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0150142d-7c7a-4413-acd3-b3b196107d58-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh\" (UID: \"0150142d-7c7a-4413-acd3-b3b196107d58\") " pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:06.177728 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.177715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ff142c-a911-46bf-b471-7825d327add4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz\" (UID: \"13ff142c-a911-46bf-b471-7825d327add4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:06.185073 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.185049    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n94pv\" (UniqueName: \"kubernetes.io/projected/0150142d-7c7a-4413-acd3-b3b196107d58-kube-api-access-n94pv\") pod \"payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh\" (UID: \"0150142d-7c7a-4413-acd3-b3b196107d58\") " pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:06.185073 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.185070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8mrt\" (UniqueName: \"kubernetes.io/projected/13ff142c-a911-46bf-b471-7825d327add4-kube-api-access-g8mrt\") pod \"payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz\" (UID: \"13ff142c-a911-46bf-b471-7825d327add4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:06.257005 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.256968    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:06.287734 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.287693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:06.408707 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.408678    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"]
Jul 27 20:08:06.410498 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:08:06.410459    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ff142c_a911_46bf_b471_7825d327add4.slice/crio-9ab6c04e890367c7a1eb3f41b934034a9571d301ce739e9f7960afb9f7d5da69 WatchSource:0}: Error finding container 9ab6c04e890367c7a1eb3f41b934034a9571d301ce739e9f7960afb9f7d5da69: Status 404 returned error can't find the container with id 9ab6c04e890367c7a1eb3f41b934034a9571d301ce739e9f7960afb9f7d5da69
Jul 27 20:08:06.446324 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.446224    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"]
Jul 27 20:08:06.453555 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:08:06.453516    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0150142d_7c7a_4413_acd3_b3b196107d58.slice/crio-6209baef6b36c314f8d5ffdba7e5444a3ba9759137ccc0234d265a2fb04d368e WatchSource:0}: Error finding container 6209baef6b36c314f8d5ffdba7e5444a3ba9759137ccc0234d265a2fb04d368e: Status 404 returned error can't find the container with id 6209baef6b36c314f8d5ffdba7e5444a3ba9759137ccc0234d265a2fb04d368e
Jul 27 20:08:06.623918 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.623885    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh" event={"ID":"0150142d-7c7a-4413-acd3-b3b196107d58","Type":"ContainerStarted","Data":"63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0"}
Jul 27 20:08:06.624117 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.623940    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh" event={"ID":"0150142d-7c7a-4413-acd3-b3b196107d58","Type":"ContainerStarted","Data":"6209baef6b36c314f8d5ffdba7e5444a3ba9759137ccc0234d265a2fb04d368e"}
Jul 27 20:08:06.624117 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.624013    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:06.625635 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.625607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz" event={"ID":"13ff142c-a911-46bf-b471-7825d327add4","Type":"ContainerStarted","Data":"64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c"}
Jul 27 20:08:06.625761 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.625653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz" event={"ID":"13ff142c-a911-46bf-b471-7825d327add4","Type":"ContainerStarted","Data":"9ab6c04e890367c7a1eb3f41b934034a9571d301ce739e9f7960afb9f7d5da69"}
Jul 27 20:08:06.625761 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.625704    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:06.640469 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.640422    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh" podStartSLOduration=1.6404078229999999 podStartE2EDuration="1.640407823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:08:06.638916741 +0000 UTC m=+2538.730268268" watchObservedRunningTime="2026-07-27 20:08:06.640407823 +0000 UTC m=+2538.731759322"
Jul 27 20:08:06.654903 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:06.654855    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz" podStartSLOduration=1.654840018 podStartE2EDuration="1.654840018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:08:06.653100433 +0000 UTC m=+2538.744451935" watchObservedRunningTime="2026-07-27 20:08:06.654840018 +0000 UTC m=+2538.746191515"
Jul 27 20:08:09.760016 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:09.759975    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:10.061269 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:10.061136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:10.356017 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:10.355938    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:10.473902 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:10.473862    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:13.319456 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:13.319422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:13.417906 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:13.417872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:13.555235 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:13.555202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:13.978148 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:13.978099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:14.076442 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.076412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:14.081622 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.081590    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"]
Jul 27 20:08:14.085400 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.085381    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.088107 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.088085    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-e837b8-dockercfg-khlgg\""
Jul 27 20:08:14.088234 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.088111    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-e837b8\""
Jul 27 20:08:14.093886 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.093801    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"]
Jul 27 20:08:14.152967 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.152933    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp895\" (UniqueName: \"kubernetes.io/projected/26539b65-f182-444b-b41f-ccd923c9b815-kube-api-access-kp895\") pod \"payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw\" (UID: \"26539b65-f182-444b-b41f-ccd923c9b815\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.153124 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.152985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26539b65-f182-444b-b41f-ccd923c9b815-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw\" (UID: \"26539b65-f182-444b-b41f-ccd923c9b815\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.254024 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.253947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp895\" (UniqueName: \"kubernetes.io/projected/26539b65-f182-444b-b41f-ccd923c9b815-kube-api-access-kp895\") pod \"payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw\" (UID: \"26539b65-f182-444b-b41f-ccd923c9b815\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.254024 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.253992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26539b65-f182-444b-b41f-ccd923c9b815-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw\" (UID: \"26539b65-f182-444b-b41f-ccd923c9b815\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.254676 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.254656    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26539b65-f182-444b-b41f-ccd923c9b815-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw\" (UID: \"26539b65-f182-444b-b41f-ccd923c9b815\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.261997 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.261969    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp895\" (UniqueName: \"kubernetes.io/projected/26539b65-f182-444b-b41f-ccd923c9b815-kube-api-access-kp895\") pod \"payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw\" (UID: \"26539b65-f182-444b-b41f-ccd923c9b815\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.398008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.397972    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.541982 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.541902    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"]
Jul 27 20:08:14.544571 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:08:14.544544    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26539b65_f182_444b_b41f_ccd923c9b815.slice/crio-a6e15b99361519cc364250616eb1f61966ef0fb548882e9740b963bd115dd1ce WatchSource:0}: Error finding container a6e15b99361519cc364250616eb1f61966ef0fb548882e9740b963bd115dd1ce: Status 404 returned error can't find the container with id a6e15b99361519cc364250616eb1f61966ef0fb548882e9740b963bd115dd1ce
Jul 27 20:08:14.668584 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.668548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw" event={"ID":"26539b65-f182-444b-b41f-ccd923c9b815","Type":"ContainerStarted","Data":"2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf"}
Jul 27 20:08:14.668753 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.668600    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw" event={"ID":"26539b65-f182-444b-b41f-ccd923c9b815","Type":"ContainerStarted","Data":"a6e15b99361519cc364250616eb1f61966ef0fb548882e9740b963bd115dd1ce"}
Jul 27 20:08:14.668753 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.668664    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:14.685280 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:14.685223    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw" podStartSLOduration=0.685209439 podStartE2EDuration="685.209439ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:08:14.684181459 +0000 UTC m=+2546.775532956" watchObservedRunningTime="2026-07-27 20:08:14.685209439 +0000 UTC m=+2546.776560937"
Jul 27 20:08:17.633090 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:17.633053    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:08:17.633631 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:17.633142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:08:17.796478 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:17.796444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:08:25.678085 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:25.678049    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:08:53.317114 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:08:53.317083    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:09:01.432582 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:01.432550    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:09:06.102792 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.102757    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"]
Jul 27 20:09:06.107053 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.107034    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.109606 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.109585    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"openshift-service-ca.crt\""
Jul 27 20:09:06.110787 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.110649    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"auth-test-model-97e28d-kserve-self-signed-certs\""
Jul 27 20:09:06.110787 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.110665    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"default-dockercfg-trncw\""
Jul 27 20:09:06.110787 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.110688    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"kube-root-ca.crt\""
Jul 27 20:09:06.117646 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.117618    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"]
Jul 27 20:09:06.148519 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.148488    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:09:06.246377 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.246342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tls-certs\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.246377 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.246379    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tmp-dir\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.246598 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.246508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-model-cache\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.246598 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.246539    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4j5l\" (UniqueName: \"kubernetes.io/projected/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-kube-api-access-j4j5l\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.246598 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.246574    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-home\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.246707 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.246646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-dshm\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.347605 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.347568    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tls-certs\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.347605 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.347606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tmp-dir\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.347820 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.347689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-model-cache\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.347820 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.347713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4j5l\" (UniqueName: \"kubernetes.io/projected/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-kube-api-access-j4j5l\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.347820 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.347753    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-home\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.347820 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.347794    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-dshm\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.348133 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.348100    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-model-cache\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.348291 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.348142    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tmp-dir\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.348291 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.348211    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-home\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.350268 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.350209    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-dshm\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.350556 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.350532    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tls-certs\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.356117 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.356066    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4j5l\" (UniqueName: \"kubernetes.io/projected/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-kube-api-access-j4j5l\") pod \"auth-test-model-97e28d-kserve-774f4fbc9f-hltqv\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") " pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.422130 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.422097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:06.557935 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.557910    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"]
Jul 27 20:09:06.560284 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:09:06.560227    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3de81ce0_c0f3_4ec2_8836_a1d9c86748c4.slice/crio-82098b83ab6651f66216fd94e7438606844bec9f14efa8d192a91a58d1189323 WatchSource:0}: Error finding container 82098b83ab6651f66216fd94e7438606844bec9f14efa8d192a91a58d1189323: Status 404 returned error can't find the container with id 82098b83ab6651f66216fd94e7438606844bec9f14efa8d192a91a58d1189323
Jul 27 20:09:06.938720 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.938685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv" event={"ID":"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4","Type":"ContainerStarted","Data":"217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae"}
Jul 27 20:09:06.938720 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.938728    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv" event={"ID":"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4","Type":"ContainerStarted","Data":"82098b83ab6651f66216fd94e7438606844bec9f14efa8d192a91a58d1189323"}
Jul 27 20:09:06.955077 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:06.955009    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv" podStartSLOduration=0.954992921 podStartE2EDuration="954.992921ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:09:06.95402964 +0000 UTC m=+2599.045381137" watchObservedRunningTime="2026-07-27 20:09:06.954992921 +0000 UTC m=+2599.046344420"
Jul 27 20:09:16.422352 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:16.422315    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:16.422352 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:16.422363    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:16.435087 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:16.435063    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:17.003283 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:17.003254    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:09:39.884479 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:09:39.884444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:10:04.393831 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:10:04.393798    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:10:46.068866 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:10:46.068778    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:10:48.556934 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:10:48.556908    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:10:48.558203 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:10:48.558181    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:10:48.565297 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:10:48.565280    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:10:48.566284 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:10:48.566269    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:12:14.917885 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:12:14.917797    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:12:14.983532 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:12:14.983476    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:12:48.735708 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:12:48.735676    2579 scope.go:117] "RemoveContainer" containerID="167509026d7f4d9568cae8f62b1e21cf8cf7961cfc8993aed2fcda7ea802e063"
Jul 27 20:12:48.744283 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:12:48.744260    2579 scope.go:117] "RemoveContainer" containerID="54b1945801c4bcf0b315da12667aef0412c3fa4f9265023dae216e99151a9a59"
Jul 27 20:13:48.769399 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:13:48.769320    2579 scope.go:117] "RemoveContainer" containerID="cd32cc8cd0c4be6dc3a6e1495734c3ef1cc9ac9cfa1d7b96f974bdbd1628d5a2"
Jul 27 20:14:11.026220 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:11.026173    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"]
Jul 27 20:14:11.074259 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:11.074177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:14:11.895194 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:11.895154    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:14:26.048555 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.048487    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv" podUID="3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" containerName="main" containerID="cri-o://217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae" gracePeriod=45
Jul 27 20:14:26.304946 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.304886    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:14:26.433501 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433466    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4j5l\" (UniqueName: \"kubernetes.io/projected/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-kube-api-access-j4j5l\") pod \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") "
Jul 27 20:14:26.433694 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433533    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-model-cache\") pod \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") "
Jul 27 20:14:26.433694 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433608    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tmp-dir\") pod \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") "
Jul 27 20:14:26.433694 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433641    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tls-certs\") pod \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") "
Jul 27 20:14:26.433694 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-home\") pod \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") "
Jul 27 20:14:26.433927 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433747    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-dshm\") pod \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\" (UID: \"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4\") "
Jul 27 20:14:26.433927 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433786    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-model-cache" (OuterVolumeSpecName: "model-cache") pod "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" (UID: "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:26.434034 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433897    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" (UID: "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:26.434034 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.433962    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-home" (OuterVolumeSpecName: "home") pod "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" (UID: "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:26.434156 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.434086    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-model-cache\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:14:26.434156 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.434106    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tmp-dir\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:14:26.434156 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.434119    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-home\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:14:26.436008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.435984    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-dshm" (OuterVolumeSpecName: "dshm") pod "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" (UID: "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:26.436180 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.436163    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" (UID: "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:14:26.436231 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.436197    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-kube-api-access-j4j5l" (OuterVolumeSpecName: "kube-api-access-j4j5l") pod "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" (UID: "3de81ce0-c0f3-4ec2-8836-a1d9c86748c4"). InnerVolumeSpecName "kube-api-access-j4j5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:14:26.535500 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.535467    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-dshm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:14:26.535500 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.535494    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4j5l\" (UniqueName: \"kubernetes.io/projected/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-kube-api-access-j4j5l\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:14:26.535698 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.535507    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4-tls-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:14:26.580750 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.580670    2579 generic.go:356] "Generic (PLEG): container finished" podID="3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" containerID="217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae" exitCode=0
Jul 27 20:14:26.580750 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.580705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv" event={"ID":"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4","Type":"ContainerDied","Data":"217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae"}
Jul 27 20:14:26.580750 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.580717    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"
Jul 27 20:14:26.580750 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.580733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv" event={"ID":"3de81ce0-c0f3-4ec2-8836-a1d9c86748c4","Type":"ContainerDied","Data":"82098b83ab6651f66216fd94e7438606844bec9f14efa8d192a91a58d1189323"}
Jul 27 20:14:26.581050 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.580756    2579 scope.go:117] "RemoveContainer" containerID="217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae"
Jul 27 20:14:26.589560 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.589530    2579 scope.go:117] "RemoveContainer" containerID="217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae"
Jul 27 20:14:26.589811 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:14:26.589790    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae\": container with ID starting with 217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae not found: ID does not exist" containerID="217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae"
Jul 27 20:14:26.589852 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.589820    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae"} err="failed to get container status \"217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae\": rpc error: code = NotFound desc = could not find container \"217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae\": container with ID starting with 217daff007dec941447a9b9d310db4e78e8857d866d2ff6abff19ef9eb2069ae not found: ID does not exist"
Jul 27 20:14:26.596968 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.596945    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"]
Jul 27 20:14:26.600401 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:26.600379    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/auth-test-model-97e28d-kserve-774f4fbc9f-hltqv"]
Jul 27 20:14:28.489132 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:28.489097    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" path="/var/lib/kubelet/pods/3de81ce0-c0f3-4ec2-8836-a1d9c86748c4/volumes"
Jul 27 20:14:29.796008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.795970    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"]
Jul 27 20:14:29.796605 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.796585    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" containerName="main"
Jul 27 20:14:29.796721 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.796607    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" containerName="main"
Jul 27 20:14:29.796780 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.796739    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3de81ce0-c0f3-4ec2-8836-a1d9c86748c4" containerName="main"
Jul 27 20:14:29.799861 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.799840    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:29.802210 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.802191    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"kube-root-ca.crt\""
Jul 27 20:14:29.803086 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.803064    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"openshift-service-ca.crt\""
Jul 27 20:14:29.803188 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.803128    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"default-dockercfg-trncw\""
Jul 27 20:14:29.803436 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.803421    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"sub-test-model-97e28d-kserve-self-signed-certs\""
Jul 27 20:14:29.807919 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.807898    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"]
Jul 27 20:14:29.830101 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.830074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:14:29.967703 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.967644    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tls-certs\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:29.967703 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.967712    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwtrt\" (UniqueName: \"kubernetes.io/projected/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-kube-api-access-mwtrt\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:29.967930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.967776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-home\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:29.967930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.967835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tmp-dir\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:29.967930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.967894    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-dshm\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:29.967930 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:29.967920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-model-cache\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.068546 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.068448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-dshm\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.068546 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.068490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-model-cache\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.068546 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.068544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tls-certs\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.068843 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.068567    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwtrt\" (UniqueName: \"kubernetes.io/projected/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-kube-api-access-mwtrt\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.068843 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.068585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-home\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.068843 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.068622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tmp-dir\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.069078 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.069051    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-home\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.069294 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.069103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tmp-dir\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.069294 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.069129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-model-cache\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.070847 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.070828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-dshm\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.071107 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.071090    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tls-certs\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.075667 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.075638    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwtrt\" (UniqueName: \"kubernetes.io/projected/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-kube-api-access-mwtrt\") pod \"sub-test-model-97e28d-kserve-7c75f59c57-vbnf6\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") " pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.111872 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.111843    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:30.241321 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.241296    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"]
Jul 27 20:14:30.243111 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:14:30.243081    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod604e6f08_ca40_41db_ae4b_7d26d2fc5d6e.slice/crio-01210d4c7acc10cf145c90c728308f8df3421993c18526cdfbe15e5ff3b3fdb7 WatchSource:0}: Error finding container 01210d4c7acc10cf145c90c728308f8df3421993c18526cdfbe15e5ff3b3fdb7: Status 404 returned error can't find the container with id 01210d4c7acc10cf145c90c728308f8df3421993c18526cdfbe15e5ff3b3fdb7
Jul 27 20:14:30.610354 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.610317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6" event={"ID":"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e","Type":"ContainerStarted","Data":"f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4"}
Jul 27 20:14:30.610560 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.610363    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6" event={"ID":"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e","Type":"ContainerStarted","Data":"01210d4c7acc10cf145c90c728308f8df3421993c18526cdfbe15e5ff3b3fdb7"}
Jul 27 20:14:30.626561 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:30.626494    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6" podStartSLOduration=1.6264764280000001 podStartE2EDuration="1.626476428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:14:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:14:30.625746212 +0000 UTC m=+2922.717097722" watchObservedRunningTime="2026-07-27 20:14:30.626476428 +0000 UTC m=+2922.717827927"
Jul 27 20:14:40.112855 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:40.112824    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:40.113280 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:40.112890    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:40.125594 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:40.125566    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:14:40.672927 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:14:40.672899    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:15:00.167640 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.167597    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"]
Jul 27 20:15:00.171316 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.171293    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"
Jul 27 20:15:00.175080 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.175061    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 20:15:00.180709 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.180693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"
Jul 27 20:15:00.203306 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.203211    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"]
Jul 27 20:15:00.325004 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.324983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"]
Jul 27 20:15:00.327533 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:15:00.327501    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091ac799_85a2_4746_be05_2935b696afa8.slice/crio-66a87ceb7dbe47a6582afbb93ede303a78e0b7f87004b5e3203862041e2c0e87 WatchSource:0}: Error finding container 66a87ceb7dbe47a6582afbb93ede303a78e0b7f87004b5e3203862041e2c0e87: Status 404 returned error can't find the container with id 66a87ceb7dbe47a6582afbb93ede303a78e0b7f87004b5e3203862041e2c0e87
Jul 27 20:15:00.764903 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.764870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb" event={"ID":"091ac799-85a2-4746-be05-2935b696afa8","Type":"ContainerStarted","Data":"86fdd2acb5bc48696ca604eace8d2e086f051ba51c9941697ff7321a114e62ad"}
Jul 27 20:15:00.765081 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.764932    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb" event={"ID":"091ac799-85a2-4746-be05-2935b696afa8","Type":"ContainerStarted","Data":"66a87ceb7dbe47a6582afbb93ede303a78e0b7f87004b5e3203862041e2c0e87"}
Jul 27 20:15:00.780937 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:00.780887    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb" podStartSLOduration=0.780870461 podStartE2EDuration="780.870461ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:15:00.778452079 +0000 UTC m=+2952.869803585" watchObservedRunningTime="2026-07-27 20:15:00.780870461 +0000 UTC m=+2952.872221991"
Jul 27 20:15:01.772053 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:01.772019    2579 generic.go:356] "Generic (PLEG): container finished" podID="091ac799-85a2-4746-be05-2935b696afa8" containerID="86fdd2acb5bc48696ca604eace8d2e086f051ba51c9941697ff7321a114e62ad" exitCode=0
Jul 27 20:15:01.772474 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:01.772095    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb" event={"ID":"091ac799-85a2-4746-be05-2935b696afa8","Type":"ContainerDied","Data":"86fdd2acb5bc48696ca604eace8d2e086f051ba51c9941697ff7321a114e62ad"}
Jul 27 20:15:02.913952 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:02.913928    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"
Jul 27 20:15:03.784535 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:03.784505    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"
Jul 27 20:15:03.784714 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:03.784532    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb" event={"ID":"091ac799-85a2-4746-be05-2935b696afa8","Type":"ContainerDied","Data":"66a87ceb7dbe47a6582afbb93ede303a78e0b7f87004b5e3203862041e2c0e87"}
Jul 27 20:15:03.784714 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:03.784582    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66a87ceb7dbe47a6582afbb93ede303a78e0b7f87004b5e3203862041e2c0e87"
Jul 27 20:15:04.319847 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:04.319803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:15:48.597655 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:48.597625    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:15:48.598755 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:48.598735    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:15:48.605527 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:48.605509    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:15:48.606464 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:48.606450    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:15:51.608435 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:15:51.608402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:20.959332 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:20.959295    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"]
Jul 27 20:16:20.976873 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:20.976845    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:21.870711 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:21.870675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:29.374597 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:29.374562    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:34.352444 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:34.352409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:35.978681 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:35.978638    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6" podUID="604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" containerName="main" containerID="cri-o://f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4" gracePeriod=45
Jul 27 20:16:36.230335 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.230264    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:16:36.255821 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.255788    2579 generic.go:356] "Generic (PLEG): container finished" podID="604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" containerID="f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4" exitCode=0
Jul 27 20:16:36.255821 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.255825    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"
Jul 27 20:16:36.256023 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.255868    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6" event={"ID":"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e","Type":"ContainerDied","Data":"f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4"}
Jul 27 20:16:36.256023 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.255910    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6" event={"ID":"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e","Type":"ContainerDied","Data":"01210d4c7acc10cf145c90c728308f8df3421993c18526cdfbe15e5ff3b3fdb7"}
Jul 27 20:16:36.256023 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.255930    2579 scope.go:117] "RemoveContainer" containerID="f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4"
Jul 27 20:16:36.266924 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.266912    2579 scope.go:117] "RemoveContainer" containerID="f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4"
Jul 27 20:16:36.267229 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:16:36.267209    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4\": container with ID starting with f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4 not found: ID does not exist" containerID="f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4"
Jul 27 20:16:36.267313 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.267237    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4"} err="failed to get container status \"f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4\": rpc error: code = NotFound desc = could not find container \"f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4\": container with ID starting with f30bce9ffd1db3eaedc99a532cfb4d26418c40b096ad4b7e47f22ca7865e44a4 not found: ID does not exist"
Jul 27 20:16:36.376722 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.376686    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-home\") pod \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") "
Jul 27 20:16:36.376899 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.376748    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tls-certs\") pod \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") "
Jul 27 20:16:36.376899 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.376775    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tmp-dir\") pod \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") "
Jul 27 20:16:36.376899 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.376795    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-dshm\") pod \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") "
Jul 27 20:16:36.376899 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.376809    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-model-cache\") pod \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") "
Jul 27 20:16:36.376899 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.376836    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwtrt\" (UniqueName: \"kubernetes.io/projected/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-kube-api-access-mwtrt\") pod \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\" (UID: \"604e6f08-ca40-41db-ae4b-7d26d2fc5d6e\") "
Jul 27 20:16:36.377164 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.377017    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-home" (OuterVolumeSpecName: "home") pod "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" (UID: "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:36.377164 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.377082    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-model-cache" (OuterVolumeSpecName: "model-cache") pod "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" (UID: "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:36.377164 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.377101    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" (UID: "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:36.377339 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.377274    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-home\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:16:36.377339 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.377302    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tmp-dir\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:16:36.377339 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.377315    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-model-cache\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:16:36.378995 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.378967    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" (UID: "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:16:36.378995 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.378977    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-dshm" (OuterVolumeSpecName: "dshm") pod "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" (UID: "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:36.379141 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.379029    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-kube-api-access-mwtrt" (OuterVolumeSpecName: "kube-api-access-mwtrt") pod "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" (UID: "604e6f08-ca40-41db-ae4b-7d26d2fc5d6e"). InnerVolumeSpecName "kube-api-access-mwtrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:36.478147 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.478107    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-tls-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:16:36.478147 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.478140    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-dshm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:16:36.478147 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.478154    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwtrt\" (UniqueName: \"kubernetes.io/projected/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e-kube-api-access-mwtrt\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:16:36.572088 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.572032    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"]
Jul 27 20:16:36.574753 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:36.574730    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/sub-test-model-97e28d-kserve-7c75f59c57-vbnf6"]
Jul 27 20:16:38.489023 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:38.488981    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" path="/var/lib/kubelet/pods/604e6f08-ca40-41db-ae4b-7d26d2fc5d6e/volumes"
Jul 27 20:16:47.413707 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.413628    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"]
Jul 27 20:16:47.414070 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.414050    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" containerName="main"
Jul 27 20:16:47.414070 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.414060    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" containerName="main"
Jul 27 20:16:47.414141 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.414080    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091ac799-85a2-4746-be05-2935b696afa8" containerName="cleanup"
Jul 27 20:16:47.414141 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.414086    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="091ac799-85a2-4746-be05-2935b696afa8" containerName="cleanup"
Jul 27 20:16:47.414205 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.414155    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="604e6f08-ca40-41db-ae4b-7d26d2fc5d6e" containerName="main"
Jul 27 20:16:47.414205 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.414166    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="091ac799-85a2-4746-be05-2935b696afa8" containerName="cleanup"
Jul 27 20:16:47.418991 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.418972    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.421497 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.421474    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"rate-test-model-97e28d-kserve-self-signed-certs\""
Jul 27 20:16:47.422448 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.422428    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"kube-root-ca.crt\""
Jul 27 20:16:47.422583 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.422475    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"default-dockercfg-trncw\""
Jul 27 20:16:47.422583 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.422487    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-97e28d\"/\"openshift-service-ca.crt\""
Jul 27 20:16:47.427535 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.427356    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"]
Jul 27 20:16:47.429094 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.429067    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:16:47.468037 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.467999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-dshm\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.468037 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.468038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f18146-660b-4f87-9e19-9cfc0dd72f59-tls-certs\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.468315 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.468087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-home\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.468315 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.468111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-tmp-dir\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.468315 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.468214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4blth\" (UniqueName: \"kubernetes.io/projected/44f18146-660b-4f87-9e19-9cfc0dd72f59-kube-api-access-4blth\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.468315 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.468282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-model-cache\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569428 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569386    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-home\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569682 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-tmp-dir\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569682 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4blth\" (UniqueName: \"kubernetes.io/projected/44f18146-660b-4f87-9e19-9cfc0dd72f59-kube-api-access-4blth\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569682 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-model-cache\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569682 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-dshm\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569682 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f18146-660b-4f87-9e19-9cfc0dd72f59-tls-certs\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569959 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-home\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569959 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569849    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-tmp-dir\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.569959 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.569955    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-model-cache\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.571917 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.571894    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-dshm\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.572023 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.572012    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f18146-660b-4f87-9e19-9cfc0dd72f59-tls-certs\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.577361 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.577340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4blth\" (UniqueName: \"kubernetes.io/projected/44f18146-660b-4f87-9e19-9cfc0dd72f59-kube-api-access-4blth\") pod \"rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") " pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.731147 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.731113    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:47.867778 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:47.867754    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"]
Jul 27 20:16:47.869403 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:16:47.869377    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44f18146_660b_4f87_9e19_9cfc0dd72f59.slice/crio-690d6358417171114126660a463095237d00c01856fc8ff5ab54488035d34ce9 WatchSource:0}: Error finding container 690d6358417171114126660a463095237d00c01856fc8ff5ab54488035d34ce9: Status 404 returned error can't find the container with id 690d6358417171114126660a463095237d00c01856fc8ff5ab54488035d34ce9
Jul 27 20:16:48.319202 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:48.319169    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl" event={"ID":"44f18146-660b-4f87-9e19-9cfc0dd72f59","Type":"ContainerStarted","Data":"c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d"}
Jul 27 20:16:48.319381 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:48.319214    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl" event={"ID":"44f18146-660b-4f87-9e19-9cfc0dd72f59","Type":"ContainerStarted","Data":"690d6358417171114126660a463095237d00c01856fc8ff5ab54488035d34ce9"}
Jul 27 20:16:48.341478 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:48.341431    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl" podStartSLOduration=1.341417516 podStartE2EDuration="1.341417516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:16:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:16:48.339180566 +0000 UTC m=+3060.430532065" watchObservedRunningTime="2026-07-27 20:16:48.341417516 +0000 UTC m=+3060.432769057"
Jul 27 20:16:57.731484 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:57.731437    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:57.732008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:57.731504    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:57.744088 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:57.744062    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:16:58.382703 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:16:58.382667    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:17:06.391643 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.391604    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"]
Jul 27 20:17:06.395494 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.395466    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.399716 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.399691    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e837b8\"/\"rate-test-model-e837b8-kserve-self-signed-certs\""
Jul 27 20:17:06.399851 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.399694    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e837b8\"/\"kube-root-ca.crt\""
Jul 27 20:17:06.399851 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.399748    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e837b8\"/\"openshift-service-ca.crt\""
Jul 27 20:17:06.399980 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.399748    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e837b8\"/\"default-dockercfg-fwmk2\""
Jul 27 20:17:06.404191 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.404145    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"]
Jul 27 20:17:06.424017 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.423993    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:17:06.455088 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.455060    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-home\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.455286 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.455103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-model-cache\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.455286 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.455180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhj2g\" (UniqueName: \"kubernetes.io/projected/90b346f5-9385-4069-9341-6c4b5194710a-kube-api-access-rhj2g\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.455286 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.455209    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-dshm\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.455286 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.455225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-tmp-dir\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.455447 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.455323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b346f5-9385-4069-9341-6c4b5194710a-tls-certs\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.556480 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.556442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhj2g\" (UniqueName: \"kubernetes.io/projected/90b346f5-9385-4069-9341-6c4b5194710a-kube-api-access-rhj2g\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.556657 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.556489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-dshm\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.556657 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.556539    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-tmp-dir\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.556748 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.556710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b346f5-9385-4069-9341-6c4b5194710a-tls-certs\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.556842 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.556823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-home\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.556908 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.556893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-model-cache\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.557015 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.556992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-tmp-dir\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.557165 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.557146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-home\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.557274 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.557198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-model-cache\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.558921 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.558896    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-dshm\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.559339 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.559320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b346f5-9385-4069-9341-6c4b5194710a-tls-certs\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.563936 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.563914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhj2g\" (UniqueName: \"kubernetes.io/projected/90b346f5-9385-4069-9341-6c4b5194710a-kube-api-access-rhj2g\") pod \"rate-test-model-e837b8-kserve-59d85ff48c-gnmbm\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") " pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.707803 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.707770    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:06.839968 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:06.839942    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"]
Jul 27 20:17:06.842125 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:17:06.842083    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90b346f5_9385_4069_9341_6c4b5194710a.slice/crio-d6938b53a558e34fa70522065ee41be61db8ef776a742a927fa245344edb89c5 WatchSource:0}: Error finding container d6938b53a558e34fa70522065ee41be61db8ef776a742a927fa245344edb89c5: Status 404 returned error can't find the container with id d6938b53a558e34fa70522065ee41be61db8ef776a742a927fa245344edb89c5
Jul 27 20:17:07.422396 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:07.422350    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm" event={"ID":"90b346f5-9385-4069-9341-6c4b5194710a","Type":"ContainerStarted","Data":"805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5"}
Jul 27 20:17:07.422396 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:07.422399    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm" event={"ID":"90b346f5-9385-4069-9341-6c4b5194710a","Type":"ContainerStarted","Data":"d6938b53a558e34fa70522065ee41be61db8ef776a742a927fa245344edb89c5"}
Jul 27 20:17:07.440434 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:07.440383    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm" podStartSLOduration=1.44036884 podStartE2EDuration="1.44036884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:17:07.439529563 +0000 UTC m=+3079.530881061" watchObservedRunningTime="2026-07-27 20:17:07.44036884 +0000 UTC m=+3079.531720337"
Jul 27 20:17:16.708869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:16.708827    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:16.711195 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:16.708891    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:16.721744 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:16.721718    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:17:17.494202 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:17:17.494171    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:18:14.140374 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:14.140283    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"]
Jul 27 20:18:14.161065 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:14.161030    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:14.998973 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:14.998936    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"]
Jul 27 20:18:15.017121 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:15.017091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:22.164640 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:22.164605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:29.159562 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.159496    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl" podUID="44f18146-660b-4f87-9e19-9cfc0dd72f59" containerName="main" containerID="cri-o://c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d" gracePeriod=45
Jul 27 20:18:29.406013 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.405988    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:18:29.574234 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574200    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f18146-660b-4f87-9e19-9cfc0dd72f59-tls-certs\") pod \"44f18146-660b-4f87-9e19-9cfc0dd72f59\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") "
Jul 27 20:18:29.574458 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574272    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-dshm\") pod \"44f18146-660b-4f87-9e19-9cfc0dd72f59\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") "
Jul 27 20:18:29.574458 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574331    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4blth\" (UniqueName: \"kubernetes.io/projected/44f18146-660b-4f87-9e19-9cfc0dd72f59-kube-api-access-4blth\") pod \"44f18146-660b-4f87-9e19-9cfc0dd72f59\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") "
Jul 27 20:18:29.574458 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574376    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-model-cache\") pod \"44f18146-660b-4f87-9e19-9cfc0dd72f59\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") "
Jul 27 20:18:29.574458 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574440    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-tmp-dir\") pod \"44f18146-660b-4f87-9e19-9cfc0dd72f59\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") "
Jul 27 20:18:29.574686 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574533    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-home\") pod \"44f18146-660b-4f87-9e19-9cfc0dd72f59\" (UID: \"44f18146-660b-4f87-9e19-9cfc0dd72f59\") "
Jul 27 20:18:29.574771 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-model-cache" (OuterVolumeSpecName: "model-cache") pod "44f18146-660b-4f87-9e19-9cfc0dd72f59" (UID: "44f18146-660b-4f87-9e19-9cfc0dd72f59"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:29.574826 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574792    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44f18146-660b-4f87-9e19-9cfc0dd72f59" (UID: "44f18146-660b-4f87-9e19-9cfc0dd72f59"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:29.574877 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574831    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-home" (OuterVolumeSpecName: "home") pod "44f18146-660b-4f87-9e19-9cfc0dd72f59" (UID: "44f18146-660b-4f87-9e19-9cfc0dd72f59"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:29.574965 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574947    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-tmp-dir\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:29.575028 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.574969    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-model-cache\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:29.576530 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.576505    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-dshm" (OuterVolumeSpecName: "dshm") pod "44f18146-660b-4f87-9e19-9cfc0dd72f59" (UID: "44f18146-660b-4f87-9e19-9cfc0dd72f59"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:29.576530 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.576520    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f18146-660b-4f87-9e19-9cfc0dd72f59-kube-api-access-4blth" (OuterVolumeSpecName: "kube-api-access-4blth") pod "44f18146-660b-4f87-9e19-9cfc0dd72f59" (UID: "44f18146-660b-4f87-9e19-9cfc0dd72f59"). InnerVolumeSpecName "kube-api-access-4blth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:29.576686 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.576592    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44f18146-660b-4f87-9e19-9cfc0dd72f59-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44f18146-660b-4f87-9e19-9cfc0dd72f59" (UID: "44f18146-660b-4f87-9e19-9cfc0dd72f59"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:18:29.676038 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.676004    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-home\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:29.676038 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.676032    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f18146-660b-4f87-9e19-9cfc0dd72f59-tls-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:29.676038 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.676042    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f18146-660b-4f87-9e19-9cfc0dd72f59-dshm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:29.676312 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.676052    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4blth\" (UniqueName: \"kubernetes.io/projected/44f18146-660b-4f87-9e19-9cfc0dd72f59-kube-api-access-4blth\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:29.854883 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.854796    2579 generic.go:356] "Generic (PLEG): container finished" podID="44f18146-660b-4f87-9e19-9cfc0dd72f59" containerID="c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d" exitCode=0
Jul 27 20:18:29.854883 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.854832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl" event={"ID":"44f18146-660b-4f87-9e19-9cfc0dd72f59","Type":"ContainerDied","Data":"c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d"}
Jul 27 20:18:29.854883 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.854841    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"
Jul 27 20:18:29.854883 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.854862    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl" event={"ID":"44f18146-660b-4f87-9e19-9cfc0dd72f59","Type":"ContainerDied","Data":"690d6358417171114126660a463095237d00c01856fc8ff5ab54488035d34ce9"}
Jul 27 20:18:29.855285 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.854886    2579 scope.go:117] "RemoveContainer" containerID="c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d"
Jul 27 20:18:29.864385 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.864370    2579 scope.go:117] "RemoveContainer" containerID="c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d"
Jul 27 20:18:29.864609 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:18:29.864590    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d\": container with ID starting with c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d not found: ID does not exist" containerID="c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d"
Jul 27 20:18:29.864657 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.864616    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d"} err="failed to get container status \"c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d\": rpc error: code = NotFound desc = could not find container \"c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d\": container with ID starting with c80473f2af3c72b3cc89e0bf29fd38e9910b9c18e95f3ae28c023eff3991958d not found: ID does not exist"
Jul 27 20:18:29.876505 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.876482    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"]
Jul 27 20:18:29.878355 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:29.878335    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-97e28d/rate-test-model-97e28d-kserve-54d94cf7c5-zd8vl"]
Jul 27 20:18:30.018419 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.018351    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm" podUID="90b346f5-9385-4069-9341-6c4b5194710a" containerName="main" containerID="cri-o://805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5" gracePeriod=45
Jul 27 20:18:30.276567 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.276543    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:18:30.382127 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382092    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-home\") pod \"90b346f5-9385-4069-9341-6c4b5194710a\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") "
Jul 27 20:18:30.382338 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382143    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b346f5-9385-4069-9341-6c4b5194710a-tls-certs\") pod \"90b346f5-9385-4069-9341-6c4b5194710a\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") "
Jul 27 20:18:30.382338 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-model-cache\") pod \"90b346f5-9385-4069-9341-6c4b5194710a\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") "
Jul 27 20:18:30.382338 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382187    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-dshm\") pod \"90b346f5-9385-4069-9341-6c4b5194710a\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") "
Jul 27 20:18:30.382338 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382268    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhj2g\" (UniqueName: \"kubernetes.io/projected/90b346f5-9385-4069-9341-6c4b5194710a-kube-api-access-rhj2g\") pod \"90b346f5-9385-4069-9341-6c4b5194710a\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") "
Jul 27 20:18:30.382338 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382289    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-tmp-dir\") pod \"90b346f5-9385-4069-9341-6c4b5194710a\" (UID: \"90b346f5-9385-4069-9341-6c4b5194710a\") "
Jul 27 20:18:30.382618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382422    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-model-cache" (OuterVolumeSpecName: "model-cache") pod "90b346f5-9385-4069-9341-6c4b5194710a" (UID: "90b346f5-9385-4069-9341-6c4b5194710a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:30.382618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382459    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-home" (OuterVolumeSpecName: "home") pod "90b346f5-9385-4069-9341-6c4b5194710a" (UID: "90b346f5-9385-4069-9341-6c4b5194710a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:30.382721 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382651    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-home\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:30.382721 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382672    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-model-cache\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:30.382721 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.382652    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "90b346f5-9385-4069-9341-6c4b5194710a" (UID: "90b346f5-9385-4069-9341-6c4b5194710a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:30.384489 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.384425    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-dshm" (OuterVolumeSpecName: "dshm") pod "90b346f5-9385-4069-9341-6c4b5194710a" (UID: "90b346f5-9385-4069-9341-6c4b5194710a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:30.384489 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.384440    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90b346f5-9385-4069-9341-6c4b5194710a-kube-api-access-rhj2g" (OuterVolumeSpecName: "kube-api-access-rhj2g") pod "90b346f5-9385-4069-9341-6c4b5194710a" (UID: "90b346f5-9385-4069-9341-6c4b5194710a"). InnerVolumeSpecName "kube-api-access-rhj2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:30.384489 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.384478    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90b346f5-9385-4069-9341-6c4b5194710a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "90b346f5-9385-4069-9341-6c4b5194710a" (UID: "90b346f5-9385-4069-9341-6c4b5194710a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:18:30.483800 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.483756    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b346f5-9385-4069-9341-6c4b5194710a-tls-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:30.483800 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.483791    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-dshm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:30.483800 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.483800    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhj2g\" (UniqueName: \"kubernetes.io/projected/90b346f5-9385-4069-9341-6c4b5194710a-kube-api-access-rhj2g\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:30.483800 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.483809    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b346f5-9385-4069-9341-6c4b5194710a-tmp-dir\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:30.489749 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.489718    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44f18146-660b-4f87-9e19-9cfc0dd72f59" path="/var/lib/kubelet/pods/44f18146-660b-4f87-9e19-9cfc0dd72f59/volumes"
Jul 27 20:18:30.861862 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.861825    2579 generic.go:356] "Generic (PLEG): container finished" podID="90b346f5-9385-4069-9341-6c4b5194710a" containerID="805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5" exitCode=0
Jul 27 20:18:30.862050 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.861870    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"
Jul 27 20:18:30.862050 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.861911    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm" event={"ID":"90b346f5-9385-4069-9341-6c4b5194710a","Type":"ContainerDied","Data":"805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5"}
Jul 27 20:18:30.862050 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.861965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm" event={"ID":"90b346f5-9385-4069-9341-6c4b5194710a","Type":"ContainerDied","Data":"d6938b53a558e34fa70522065ee41be61db8ef776a742a927fa245344edb89c5"}
Jul 27 20:18:30.862050 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.861985    2579 scope.go:117] "RemoveContainer" containerID="805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5"
Jul 27 20:18:30.870571 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.870553    2579 scope.go:117] "RemoveContainer" containerID="805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5"
Jul 27 20:18:30.870832 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:18:30.870814    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5\": container with ID starting with 805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5 not found: ID does not exist" containerID="805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5"
Jul 27 20:18:30.870878 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.870840    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5"} err="failed to get container status \"805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5\": rpc error: code = NotFound desc = could not find container \"805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5\": container with ID starting with 805316e6426da7a5c62ad8e63d13ef2c37915052b498daa83e02805d36573bb5 not found: ID does not exist"
Jul 27 20:18:30.877900 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.877879    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"]
Jul 27 20:18:30.883335 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.883313    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e837b8/rate-test-model-e837b8-kserve-59d85ff48c-gnmbm"]
Jul 27 20:18:30.908654 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.908631    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7498-139788215646944"]
Jul 27 20:18:30.909037 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.909026    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f18146-660b-4f87-9e19-9cfc0dd72f59" containerName="main"
Jul 27 20:18:30.909085 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.909039    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f18146-660b-4f87-9e19-9cfc0dd72f59" containerName="main"
Jul 27 20:18:30.909085 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.909053    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b346f5-9385-4069-9341-6c4b5194710a" containerName="main"
Jul 27 20:18:30.909085 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.909060    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b346f5-9385-4069-9341-6c4b5194710a" containerName="main"
Jul 27 20:18:30.909178 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.909134    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b346f5-9385-4069-9341-6c4b5194710a" containerName="main"
Jul 27 20:18:30.909178 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.909143    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f18146-660b-4f87-9e19-9cfc0dd72f59" containerName="main"
Jul 27 20:18:30.913451 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.913435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7498-139788215646944"
Jul 27 20:18:30.915874 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.915852    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zbdwp\""
Jul 27 20:18:30.917611 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.917591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7498-139788215646944"]
Jul 27 20:18:30.988983 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:30.988951    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb5b2\" (UniqueName: \"kubernetes.io/projected/bee2eef2-f34b-4e5f-bcf6-e1c7110606c6-kube-api-access-xb5b2\") pod \"test-curl-7498-139788215646944\" (UID: \"bee2eef2-f34b-4e5f-bcf6-e1c7110606c6\") " pod="opendatahub/test-curl-7498-139788215646944"
Jul 27 20:18:31.089481 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:31.089450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb5b2\" (UniqueName: \"kubernetes.io/projected/bee2eef2-f34b-4e5f-bcf6-e1c7110606c6-kube-api-access-xb5b2\") pod \"test-curl-7498-139788215646944\" (UID: \"bee2eef2-f34b-4e5f-bcf6-e1c7110606c6\") " pod="opendatahub/test-curl-7498-139788215646944"
Jul 27 20:18:31.097394 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:31.097368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb5b2\" (UniqueName: \"kubernetes.io/projected/bee2eef2-f34b-4e5f-bcf6-e1c7110606c6-kube-api-access-xb5b2\") pod \"test-curl-7498-139788215646944\" (UID: \"bee2eef2-f34b-4e5f-bcf6-e1c7110606c6\") " pod="opendatahub/test-curl-7498-139788215646944"
Jul 27 20:18:31.224115 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:31.224079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7498-139788215646944"
Jul 27 20:18:31.364044 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:31.364019    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7498-139788215646944"]
Jul 27 20:18:31.366053 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:18:31.366025    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbee2eef2_f34b_4e5f_bcf6_e1c7110606c6.slice/crio-b7e4abbc399b24c2477ebefcc8e590925f1e5259347213af5a60101b89744436 WatchSource:0}: Error finding container b7e4abbc399b24c2477ebefcc8e590925f1e5259347213af5a60101b89744436: Status 404 returned error can't find the container with id b7e4abbc399b24c2477ebefcc8e590925f1e5259347213af5a60101b89744436
Jul 27 20:18:31.367391 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:31.367372    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 20:18:31.871293 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:31.871238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7498-139788215646944" event={"ID":"bee2eef2-f34b-4e5f-bcf6-e1c7110606c6","Type":"ContainerStarted","Data":"b7e4abbc399b24c2477ebefcc8e590925f1e5259347213af5a60101b89744436"}
Jul 27 20:18:32.490723 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:32.490693    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90b346f5-9385-4069-9341-6c4b5194710a" path="/var/lib/kubelet/pods/90b346f5-9385-4069-9341-6c4b5194710a/volumes"
Jul 27 20:18:32.879813 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:32.879714    2579 generic.go:356] "Generic (PLEG): container finished" podID="bee2eef2-f34b-4e5f-bcf6-e1c7110606c6" containerID="1881a3a0bdadf1ce655410048ee8fbc0d4d1aa80b69d08099bd5b84ab7e1bbb4" exitCode=0
Jul 27 20:18:32.879813 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:32.879786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7498-139788215646944" event={"ID":"bee2eef2-f34b-4e5f-bcf6-e1c7110606c6","Type":"ContainerDied","Data":"1881a3a0bdadf1ce655410048ee8fbc0d4d1aa80b69d08099bd5b84ab7e1bbb4"}
Jul 27 20:18:34.015088 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.015063    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7498-139788215646944"
Jul 27 20:18:34.044978 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.044954    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7498-139788215646944_bee2eef2-f34b-4e5f-bcf6-e1c7110606c6/test-curl-7498-139788215646944/0.log"
Jul 27 20:18:34.093604 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.093561    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7498-139788215646944"]
Jul 27 20:18:34.099712 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.099684    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7498-139788215646944"]
Jul 27 20:18:34.118360 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.118339    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb5b2\" (UniqueName: \"kubernetes.io/projected/bee2eef2-f34b-4e5f-bcf6-e1c7110606c6-kube-api-access-xb5b2\") pod \"bee2eef2-f34b-4e5f-bcf6-e1c7110606c6\" (UID: \"bee2eef2-f34b-4e5f-bcf6-e1c7110606c6\") "
Jul 27 20:18:34.120362 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.120340    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bee2eef2-f34b-4e5f-bcf6-e1c7110606c6-kube-api-access-xb5b2" (OuterVolumeSpecName: "kube-api-access-xb5b2") pod "bee2eef2-f34b-4e5f-bcf6-e1c7110606c6" (UID: "bee2eef2-f34b-4e5f-bcf6-e1c7110606c6"). InnerVolumeSpecName "kube-api-access-xb5b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:34.219961 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.219929    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb5b2\" (UniqueName: \"kubernetes.io/projected/bee2eef2-f34b-4e5f-bcf6-e1c7110606c6-kube-api-access-xb5b2\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:18:34.489333 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.489257    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bee2eef2-f34b-4e5f-bcf6-e1c7110606c6" path="/var/lib/kubelet/pods/bee2eef2-f34b-4e5f-bcf6-e1c7110606c6/volumes"
Jul 27 20:18:34.581638 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.581598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:34.875899 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.875815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:34.891196 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.891163    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7498-139788215646944"
Jul 27 20:18:34.891387 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:34.891229    2579 scope.go:117] "RemoveContainer" containerID="1881a3a0bdadf1ce655410048ee8fbc0d4d1aa80b69d08099bd5b84ab7e1bbb4"
Jul 27 20:18:35.310117 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:35.310085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:37.544694 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:37.544656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:37.634344 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:37.634304    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:37.698273 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:37.698224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:38.030091 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.030061    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:38.239202 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.239169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:38.329233 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.329117    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:38.330665 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.330636    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"]
Jul 27 20:18:38.331080 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.331066    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bee2eef2-f34b-4e5f-bcf6-e1c7110606c6" containerName="test-curl-7498-139788215646944"
Jul 27 20:18:38.331127 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.331084    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bee2eef2-f34b-4e5f-bcf6-e1c7110606c6" containerName="test-curl-7498-139788215646944"
Jul 27 20:18:38.331180 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.331170    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bee2eef2-f34b-4e5f-bcf6-e1c7110606c6" containerName="test-curl-7498-139788215646944"
Jul 27 20:18:38.335581 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.335562    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.337973 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.337951    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-6c472a\""
Jul 27 20:18:38.344038 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.344015    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"]
Jul 27 20:18:38.376292 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.376252    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"]
Jul 27 20:18:38.379592 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.379577    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:38.382087 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.382067    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-6c472a-dockercfg-ldnnb\""
Jul 27 20:18:38.388469 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.388445    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"]
Jul 27 20:18:38.463331 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.463295    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d505f05b-734c-4ee3-b8ca-13b1b174400d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9\" (UID: \"d505f05b-734c-4ee3-b8ca-13b1b174400d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.463521 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.463447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzbbj\" (UniqueName: \"kubernetes.io/projected/d505f05b-734c-4ee3-b8ca-13b1b174400d-kube-api-access-pzbbj\") pod \"payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9\" (UID: \"d505f05b-734c-4ee3-b8ca-13b1b174400d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.564899 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.564862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d505f05b-734c-4ee3-b8ca-13b1b174400d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9\" (UID: \"d505f05b-734c-4ee3-b8ca-13b1b174400d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.565385 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.565070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls62m\" (UniqueName: \"kubernetes.io/projected/a55510e2-3e8a-4b4d-865b-d41f30c9f726-kube-api-access-ls62m\") pod \"payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9\" (UID: \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:38.565385 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.565369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzbbj\" (UniqueName: \"kubernetes.io/projected/d505f05b-734c-4ee3-b8ca-13b1b174400d-kube-api-access-pzbbj\") pod \"payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9\" (UID: \"d505f05b-734c-4ee3-b8ca-13b1b174400d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.565655 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.565628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a55510e2-3e8a-4b4d-865b-d41f30c9f726-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9\" (UID: \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:38.565655 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.565637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d505f05b-734c-4ee3-b8ca-13b1b174400d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9\" (UID: \"d505f05b-734c-4ee3-b8ca-13b1b174400d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.573392 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.573365    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzbbj\" (UniqueName: \"kubernetes.io/projected/d505f05b-734c-4ee3-b8ca-13b1b174400d-kube-api-access-pzbbj\") pod \"payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9\" (UID: \"d505f05b-734c-4ee3-b8ca-13b1b174400d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.648527 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.648448    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.666449 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.666413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a55510e2-3e8a-4b4d-865b-d41f30c9f726-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9\" (UID: \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:38.666605 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.666521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls62m\" (UniqueName: \"kubernetes.io/projected/a55510e2-3e8a-4b4d-865b-d41f30c9f726-kube-api-access-ls62m\") pod \"payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9\" (UID: \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:38.667365 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.667307    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a55510e2-3e8a-4b4d-865b-d41f30c9f726-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9\" (UID: \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:38.674696 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.674666    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls62m\" (UniqueName: \"kubernetes.io/projected/a55510e2-3e8a-4b4d-865b-d41f30c9f726-kube-api-access-ls62m\") pod \"payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9\" (UID: \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:38.690973 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.690947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:38.805182 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.805028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"]
Jul 27 20:18:38.861785 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.861761    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"]
Jul 27 20:18:38.867572 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:18:38.867545    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda55510e2_3e8a_4b4d_865b_d41f30c9f726.slice/crio-84beaab02f29e7bec784c021c61dcb15138409a5397aa4b9c0cdba30c35501db WatchSource:0}: Error finding container 84beaab02f29e7bec784c021c61dcb15138409a5397aa4b9c0cdba30c35501db: Status 404 returned error can't find the container with id 84beaab02f29e7bec784c021c61dcb15138409a5397aa4b9c0cdba30c35501db
Jul 27 20:18:38.914926 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.914901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9" event={"ID":"a55510e2-3e8a-4b4d-865b-d41f30c9f726","Type":"ContainerStarted","Data":"84beaab02f29e7bec784c021c61dcb15138409a5397aa4b9c0cdba30c35501db"}
Jul 27 20:18:38.916403 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.916383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9" event={"ID":"d505f05b-734c-4ee3-b8ca-13b1b174400d","Type":"ContainerStarted","Data":"5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0"}
Jul 27 20:18:38.916512 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.916422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9" event={"ID":"d505f05b-734c-4ee3-b8ca-13b1b174400d","Type":"ContainerStarted","Data":"a5359f54db5f0f3c46dca565c46ef5d603164ad9df24b571550701e2a53a6e8b"}
Jul 27 20:18:38.916512 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.916486    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:18:38.932166 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:38.932124    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9" podStartSLOduration=0.932106327 podStartE2EDuration="932.106327ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:18:38.931818128 +0000 UTC m=+3171.023169629" watchObservedRunningTime="2026-07-27 20:18:38.932106327 +0000 UTC m=+3171.023457826"
Jul 27 20:18:39.923543 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:39.923506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9" event={"ID":"a55510e2-3e8a-4b4d-865b-d41f30c9f726","Type":"ContainerStarted","Data":"8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860"}
Jul 27 20:18:39.923946 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:39.923589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:39.940435 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:39.940384    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9" podStartSLOduration=1.940367276 podStartE2EDuration="1.940367276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:18:39.939320258 +0000 UTC m=+3172.030671758" watchObservedRunningTime="2026-07-27 20:18:39.940367276 +0000 UTC m=+3172.031718773"
Jul 27 20:18:42.045642 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:42.045611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:18:49.925573 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:49.925529    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9" podUID="d505f05b-734c-4ee3-b8ca-13b1b174400d" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.76:9004: connect: connection refused"
Jul 27 20:18:50.931018 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:50.930984    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:18:59.924583 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:18:59.924552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:19:03.160000 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:03.159941    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:03.463185 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:03.463146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:03.634271 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:03.634226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:06.768394 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:06.768359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:06.824884 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:06.824835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:06.933140 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:06.933098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:07.393184 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.393145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:07.492342 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.492301    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:07.499871 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.499843    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"]
Jul 27 20:19:07.506274 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.505555    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:07.509522 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.509158    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d4162b\""
Jul 27 20:19:07.510059 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.510041    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d4162b-dockercfg-wl5zn\""
Jul 27 20:19:07.512394 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.512362    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"]
Jul 27 20:19:07.546403 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.546370    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"]
Jul 27 20:19:07.549035 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.549014    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:07.559980 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.559956    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"]
Jul 27 20:19:07.641181 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.641149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/611cc43a-8fd7-438a-a098-efef6f938347-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m\" (UID: \"611cc43a-8fd7-438a-a098-efef6f938347\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:07.641366 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.641295    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znd7d\" (UniqueName: \"kubernetes.io/projected/6be7f991-1cee-4d32-9fa7-f83020eb343c-kube-api-access-znd7d\") pod \"payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l\" (UID: \"6be7f991-1cee-4d32-9fa7-f83020eb343c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:07.641366 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.641334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rts\" (UniqueName: \"kubernetes.io/projected/611cc43a-8fd7-438a-a098-efef6f938347-kube-api-access-g4rts\") pod \"payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m\" (UID: \"611cc43a-8fd7-438a-a098-efef6f938347\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:07.641366 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.641354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be7f991-1cee-4d32-9fa7-f83020eb343c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l\" (UID: \"6be7f991-1cee-4d32-9fa7-f83020eb343c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:07.742869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.742832    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znd7d\" (UniqueName: \"kubernetes.io/projected/6be7f991-1cee-4d32-9fa7-f83020eb343c-kube-api-access-znd7d\") pod \"payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l\" (UID: \"6be7f991-1cee-4d32-9fa7-f83020eb343c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:07.742869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.742875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rts\" (UniqueName: \"kubernetes.io/projected/611cc43a-8fd7-438a-a098-efef6f938347-kube-api-access-g4rts\") pod \"payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m\" (UID: \"611cc43a-8fd7-438a-a098-efef6f938347\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:07.743122 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.742893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be7f991-1cee-4d32-9fa7-f83020eb343c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l\" (UID: \"6be7f991-1cee-4d32-9fa7-f83020eb343c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:07.743122 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.742974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/611cc43a-8fd7-438a-a098-efef6f938347-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m\" (UID: \"611cc43a-8fd7-438a-a098-efef6f938347\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:07.743714 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.743694    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/611cc43a-8fd7-438a-a098-efef6f938347-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m\" (UID: \"611cc43a-8fd7-438a-a098-efef6f938347\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:07.743782 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.743764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be7f991-1cee-4d32-9fa7-f83020eb343c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l\" (UID: \"6be7f991-1cee-4d32-9fa7-f83020eb343c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:07.750477 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.750453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rts\" (UniqueName: \"kubernetes.io/projected/611cc43a-8fd7-438a-a098-efef6f938347-kube-api-access-g4rts\") pod \"payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m\" (UID: \"611cc43a-8fd7-438a-a098-efef6f938347\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:07.750614 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.750522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znd7d\" (UniqueName: \"kubernetes.io/projected/6be7f991-1cee-4d32-9fa7-f83020eb343c-kube-api-access-znd7d\") pod \"payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l\" (UID: \"6be7f991-1cee-4d32-9fa7-f83020eb343c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:07.819282 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.819233    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:07.859786 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.859746    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:07.971676 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:07.971596    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"]
Jul 27 20:19:07.973832 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:19:07.973798    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be7f991_1cee_4d32_9fa7_f83020eb343c.slice/crio-30ed0462c81a6156365178b22d7427b4246c2058475fabaab26a6ece52d01481 WatchSource:0}: Error finding container 30ed0462c81a6156365178b22d7427b4246c2058475fabaab26a6ece52d01481: Status 404 returned error can't find the container with id 30ed0462c81a6156365178b22d7427b4246c2058475fabaab26a6ece52d01481
Jul 27 20:19:08.026311 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:08.026074    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"]
Jul 27 20:19:08.027722 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:19:08.027684    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611cc43a_8fd7_438a_a098_efef6f938347.slice/crio-0ec8c986c7b454b0a33dfde8ab5f69a25495ae942562de486b70e6972d7621f9 WatchSource:0}: Error finding container 0ec8c986c7b454b0a33dfde8ab5f69a25495ae942562de486b70e6972d7621f9: Status 404 returned error can't find the container with id 0ec8c986c7b454b0a33dfde8ab5f69a25495ae942562de486b70e6972d7621f9
Jul 27 20:19:08.071081 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:08.071046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l" event={"ID":"6be7f991-1cee-4d32-9fa7-f83020eb343c","Type":"ContainerStarted","Data":"9fe8417215f5041bb0231a3434aefa5c8a283a738e253ddbdd6102478b5f6190"}
Jul 27 20:19:08.071276 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:08.071117    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l" event={"ID":"6be7f991-1cee-4d32-9fa7-f83020eb343c","Type":"ContainerStarted","Data":"30ed0462c81a6156365178b22d7427b4246c2058475fabaab26a6ece52d01481"}
Jul 27 20:19:08.071405 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:08.071385    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:08.072503 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:08.072479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m" event={"ID":"611cc43a-8fd7-438a-a098-efef6f938347","Type":"ContainerStarted","Data":"0ec8c986c7b454b0a33dfde8ab5f69a25495ae942562de486b70e6972d7621f9"}
Jul 27 20:19:08.086574 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:08.086530    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l" podStartSLOduration=1.086511792 podStartE2EDuration="1.086511792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:19:08.086180635 +0000 UTC m=+3200.177532133" watchObservedRunningTime="2026-07-27 20:19:08.086511792 +0000 UTC m=+3200.177863291"
Jul 27 20:19:09.079291 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:09.079238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m" event={"ID":"611cc43a-8fd7-438a-a098-efef6f938347","Type":"ContainerStarted","Data":"4a11c1609ad54bb2b616723f9718f2676785569bb647cba33edd17b35fa9ae0e"}
Jul 27 20:19:09.079723 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:09.079460    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:09.095903 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:09.095852    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m" podStartSLOduration=2.09583716 podStartE2EDuration="2.09583716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:19:09.094911936 +0000 UTC m=+3201.186263442" watchObservedRunningTime="2026-07-27 20:19:09.09583716 +0000 UTC m=+3201.187188687"
Jul 27 20:19:11.228795 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:11.228760    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:19.081334 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:19.081303    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:19:20.086719 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:20.086689    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:19:28.376659 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:28.376624    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m_611cc43a-8fd7-438a-a098-efef6f938347/payload-processing/0.log"
Jul 27 20:19:42.100366 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.100285    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"]
Jul 27 20:19:42.103848 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.103828    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.106411 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.106392    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6c472a\"/\"default-dockercfg-pv6m4\""
Jul 27 20:19:42.107663 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.107644    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6c472a\"/\"kube-root-ca.crt\""
Jul 27 20:19:42.107767 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.107688    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6c472a\"/\"ipp-route-6c472a-kserve-self-signed-certs\""
Jul 27 20:19:42.107767 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.107694    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6c472a\"/\"openshift-service-ca.crt\""
Jul 27 20:19:42.112936 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.112916    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"]
Jul 27 20:19:42.121322 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.121292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:19:42.191219 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.191177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-home\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.191428 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.191225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tmp-dir\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.191428 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.191321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-model-cache\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.191428 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.191383    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-dshm\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.191566 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.191472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp2fx\" (UniqueName: \"kubernetes.io/projected/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-kube-api-access-dp2fx\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.191566 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.191503    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tls-certs\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292420 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-home\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tmp-dir\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-model-cache\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292513    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-dshm\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp2fx\" (UniqueName: \"kubernetes.io/projected/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-kube-api-access-dp2fx\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292599    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tls-certs\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292896 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292794    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-home\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292896 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-model-cache\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.292896 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.292893    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tmp-dir\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.294859 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.294836    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-dshm\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.295169 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.295149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tls-certs\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.300471 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.300453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp2fx\" (UniqueName: \"kubernetes.io/projected/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-kube-api-access-dp2fx\") pod \"ipp-route-6c472a-kserve-7fcb968f79-b26b6\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") " pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.414784 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.414693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:42.549316 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:42.549211    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"]
Jul 27 20:19:42.550913 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:19:42.550882    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc8baa2b_acd3_4eeb_a733_e943054d2ea5.slice/crio-9b28573f85a214291c1c70db6152d4070d2c853670e7e6d4fa9a6eabbfdf4961 WatchSource:0}: Error finding container 9b28573f85a214291c1c70db6152d4070d2c853670e7e6d4fa9a6eabbfdf4961: Status 404 returned error can't find the container with id 9b28573f85a214291c1c70db6152d4070d2c853670e7e6d4fa9a6eabbfdf4961
Jul 27 20:19:43.262480 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:43.262450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6" event={"ID":"bc8baa2b-acd3-4eeb-a733-e943054d2ea5","Type":"ContainerStarted","Data":"df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911"}
Jul 27 20:19:43.262886 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:43.262487    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6" event={"ID":"bc8baa2b-acd3-4eeb-a733-e943054d2ea5","Type":"ContainerStarted","Data":"9b28573f85a214291c1c70db6152d4070d2c853670e7e6d4fa9a6eabbfdf4961"}
Jul 27 20:19:43.279205 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:43.279161    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6" podStartSLOduration=1.279145401 podStartE2EDuration="1.279145401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:19:43.278104096 +0000 UTC m=+3235.369455595" watchObservedRunningTime="2026-07-27 20:19:43.279145401 +0000 UTC m=+3235.370496898"
Jul 27 20:19:52.415568 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:52.415527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:52.416194 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:52.415588    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:52.427890 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:52.427862    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:19:53.324095 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:19:53.324066    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:20:23.288328 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:23.288296    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9_a55510e2-3e8a-4b4d-865b-d41f30c9f726/payload-processing/0.log"
Jul 27 20:20:23.525149 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:23.525121    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m_611cc43a-8fd7-438a-a098-efef6f938347/payload-processing/0.log"
Jul 27 20:20:23.800619 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:23.800574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:24.130496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:24.130398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:24.241157 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:24.241130    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:27.175340 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:27.175295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:27.242318 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:27.242218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:27.321230 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:27.321202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:27.648190 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:27.648159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:27.751707 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:27.751654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:27.847471 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:27.847437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:28.036114 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.036079    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"]
Jul 27 20:20:28.040747 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.040720    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:28.043938 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.043916    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d4c0c3-dockercfg-sn6nb\""
Jul 27 20:20:28.044152 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.043957    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d4c0c3\""
Jul 27 20:20:28.046097 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.046073    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"]
Jul 27 20:20:28.134956 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.134919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqq48\" (UniqueName: \"kubernetes.io/projected/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-kube-api-access-bqq48\") pod \"payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6\" (UID: \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:28.135125 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.135015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6\" (UID: \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:28.236435 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.236398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqq48\" (UniqueName: \"kubernetes.io/projected/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-kube-api-access-bqq48\") pod \"payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6\" (UID: \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:28.236804 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.236464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6\" (UID: \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:28.237227 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.237208    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6\" (UID: \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:28.243855 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.243831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqq48\" (UniqueName: \"kubernetes.io/projected/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-kube-api-access-bqq48\") pod \"payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6\" (UID: \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:28.355515 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.355435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:28.496657 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:28.496634    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"]
Jul 27 20:20:28.498532 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:20:28.498505    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad01d03c_4cde_498c_88b1_fe000c8a2ca0.slice/crio-323888e80dc36293ed6ef8b5b5a941e4b39fea29429ba3c9a2efb034486223da WatchSource:0}: Error finding container 323888e80dc36293ed6ef8b5b5a941e4b39fea29429ba3c9a2efb034486223da: Status 404 returned error can't find the container with id 323888e80dc36293ed6ef8b5b5a941e4b39fea29429ba3c9a2efb034486223da
Jul 27 20:20:29.518727 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:29.518689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6" event={"ID":"ad01d03c-4cde-498c-88b1-fe000c8a2ca0","Type":"ContainerStarted","Data":"0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796"}
Jul 27 20:20:29.519150 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:29.518764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6" event={"ID":"ad01d03c-4cde-498c-88b1-fe000c8a2ca0","Type":"ContainerStarted","Data":"323888e80dc36293ed6ef8b5b5a941e4b39fea29429ba3c9a2efb034486223da"}
Jul 27 20:20:29.519150 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:29.518802    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:29.536717 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:29.536671    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6" podStartSLOduration=1.5366561490000001 podStartE2EDuration="1.536656149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:20:29.533998204 +0000 UTC m=+3281.625349702" watchObservedRunningTime="2026-07-27 20:20:29.536656149 +0000 UTC m=+3281.628007646"
Jul 27 20:20:31.648994 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:31.648951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:40.526409 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:40.526373    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:48.631851 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:48.631818    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:20:48.632860 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:48.632842    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:20:48.639100 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:48.639082    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:20:48.640107 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:48.640091    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rmqjz_68a01e15-aea1-4950-bd2a-47bac1e23587/ovnkube-controller/1.log"
Jul 27 20:20:53.142365 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.142334    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:53.162008 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.161977    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"]
Jul 27 20:20:53.162227 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.162184    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6" podUID="ad01d03c-4cde-498c-88b1-fe000c8a2ca0" containerName="payload-processing" containerID="cri-o://0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796" gracePeriod=30
Jul 27 20:20:53.423962 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.423932    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:53.595588 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.595548    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-plugins-config-volume\") pod \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\" (UID: \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\") "
Jul 27 20:20:53.595785 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.595597    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqq48\" (UniqueName: \"kubernetes.io/projected/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-kube-api-access-bqq48\") pod \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\" (UID: \"ad01d03c-4cde-498c-88b1-fe000c8a2ca0\") "
Jul 27 20:20:53.596040 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.596011    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ad01d03c-4cde-498c-88b1-fe000c8a2ca0" (UID: "ad01d03c-4cde-498c-88b1-fe000c8a2ca0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:20:53.597744 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.597711    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-kube-api-access-bqq48" (OuterVolumeSpecName: "kube-api-access-bqq48") pod "ad01d03c-4cde-498c-88b1-fe000c8a2ca0" (UID: "ad01d03c-4cde-498c-88b1-fe000c8a2ca0"). InnerVolumeSpecName "kube-api-access-bqq48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:20:53.648554 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.648519    2579 generic.go:356] "Generic (PLEG): container finished" podID="ad01d03c-4cde-498c-88b1-fe000c8a2ca0" containerID="0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796" exitCode=0
Jul 27 20:20:53.648740 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.648564    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"
Jul 27 20:20:53.648740 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.648596    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6" event={"ID":"ad01d03c-4cde-498c-88b1-fe000c8a2ca0","Type":"ContainerDied","Data":"0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796"}
Jul 27 20:20:53.648740 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.648649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6" event={"ID":"ad01d03c-4cde-498c-88b1-fe000c8a2ca0","Type":"ContainerDied","Data":"323888e80dc36293ed6ef8b5b5a941e4b39fea29429ba3c9a2efb034486223da"}
Jul 27 20:20:53.648740 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.648672    2579 scope.go:117] "RemoveContainer" containerID="0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796"
Jul 27 20:20:53.660170 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.660152    2579 scope.go:117] "RemoveContainer" containerID="0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796"
Jul 27 20:20:53.660497 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:20:53.660477    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796\": container with ID starting with 0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796 not found: ID does not exist" containerID="0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796"
Jul 27 20:20:53.660580 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.660506    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796"} err="failed to get container status \"0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796\": rpc error: code = NotFound desc = could not find container \"0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796\": container with ID starting with 0f587ad21d965d998b7401241b688b1a49f37fac8f90a31572bf25a726835796 not found: ID does not exist"
Jul 27 20:20:53.669641 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.669620    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"]
Jul 27 20:20:53.673158 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.673135    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d4c0c3-85995bc9c4-9xqf6"]
Jul 27 20:20:53.697011 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.696978    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:20:53.697011 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:53.697007    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqq48\" (UniqueName: \"kubernetes.io/projected/ad01d03c-4cde-498c-88b1-fe000c8a2ca0-kube-api-access-bqq48\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:20:54.489842 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:54.489810    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad01d03c-4cde-498c-88b1-fe000c8a2ca0" path="/var/lib/kubelet/pods/ad01d03c-4cde-498c-88b1-fe000c8a2ca0/volumes"
Jul 27 20:20:54.618546 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:54.618511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:57.661380 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:57.661346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:20:58.022659 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:20:58.022627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:03.767698 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:03.767660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:03.821112 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:03.821074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:04.064982 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:04.064910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:04.946751 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:04.946712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:10.748851 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:10.748779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:10.809605 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:10.809571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:23.166044 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.166013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:23.183572 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.183541    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"]
Jul 27 20:21:23.184690 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.183856    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9" podUID="a55510e2-3e8a-4b4d-865b-d41f30c9f726" containerName="payload-processing" containerID="cri-o://8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860" gracePeriod=30
Jul 27 20:21:23.188275 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.188234    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"]
Jul 27 20:21:23.188512 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.188477    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9" podUID="d505f05b-734c-4ee3-b8ca-13b1b174400d" containerName="payload-pre-processing" containerID="cri-o://5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0" gracePeriod=30
Jul 27 20:21:23.266638 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.266605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:23.367386 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.367340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:23.468583 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.468559    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:21:23.471547 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.471531    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:21:23.589222 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.589185    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ls62m\" (UniqueName: \"kubernetes.io/projected/a55510e2-3e8a-4b4d-865b-d41f30c9f726-kube-api-access-ls62m\") pod \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\" (UID: \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\") "
Jul 27 20:21:23.589423 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.589237    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a55510e2-3e8a-4b4d-865b-d41f30c9f726-plugins-config-volume\") pod \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\" (UID: \"a55510e2-3e8a-4b4d-865b-d41f30c9f726\") "
Jul 27 20:21:23.589423 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.589383    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzbbj\" (UniqueName: \"kubernetes.io/projected/d505f05b-734c-4ee3-b8ca-13b1b174400d-kube-api-access-pzbbj\") pod \"d505f05b-734c-4ee3-b8ca-13b1b174400d\" (UID: \"d505f05b-734c-4ee3-b8ca-13b1b174400d\") "
Jul 27 20:21:23.589563 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.589431    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d505f05b-734c-4ee3-b8ca-13b1b174400d-plugins-config-volume\") pod \"d505f05b-734c-4ee3-b8ca-13b1b174400d\" (UID: \"d505f05b-734c-4ee3-b8ca-13b1b174400d\") "
Jul 27 20:21:23.589771 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.589653    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a55510e2-3e8a-4b4d-865b-d41f30c9f726-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a55510e2-3e8a-4b4d-865b-d41f30c9f726" (UID: "a55510e2-3e8a-4b4d-865b-d41f30c9f726"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:21:23.589863 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.589841    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d505f05b-734c-4ee3-b8ca-13b1b174400d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d505f05b-734c-4ee3-b8ca-13b1b174400d" (UID: "d505f05b-734c-4ee3-b8ca-13b1b174400d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:21:23.591467 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.591433    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a55510e2-3e8a-4b4d-865b-d41f30c9f726-kube-api-access-ls62m" (OuterVolumeSpecName: "kube-api-access-ls62m") pod "a55510e2-3e8a-4b4d-865b-d41f30c9f726" (UID: "a55510e2-3e8a-4b4d-865b-d41f30c9f726"). InnerVolumeSpecName "kube-api-access-ls62m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:21:23.591467 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.591444    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d505f05b-734c-4ee3-b8ca-13b1b174400d-kube-api-access-pzbbj" (OuterVolumeSpecName: "kube-api-access-pzbbj") pod "d505f05b-734c-4ee3-b8ca-13b1b174400d" (UID: "d505f05b-734c-4ee3-b8ca-13b1b174400d"). InnerVolumeSpecName "kube-api-access-pzbbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:21:23.691056 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.691020    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls62m\" (UniqueName: \"kubernetes.io/projected/a55510e2-3e8a-4b4d-865b-d41f30c9f726-kube-api-access-ls62m\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:23.691056 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.691049    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a55510e2-3e8a-4b4d-865b-d41f30c9f726-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:23.691056 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.691059    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzbbj\" (UniqueName: \"kubernetes.io/projected/d505f05b-734c-4ee3-b8ca-13b1b174400d-kube-api-access-pzbbj\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:23.691056 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.691069    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d505f05b-734c-4ee3-b8ca-13b1b174400d-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:23.801653 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.801569    2579 generic.go:356] "Generic (PLEG): container finished" podID="a55510e2-3e8a-4b4d-865b-d41f30c9f726" containerID="8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860" exitCode=0
Jul 27 20:21:23.801653 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.801619    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"
Jul 27 20:21:23.801653 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.801619    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9" event={"ID":"a55510e2-3e8a-4b4d-865b-d41f30c9f726","Type":"ContainerDied","Data":"8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860"}
Jul 27 20:21:23.801903 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.801681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9" event={"ID":"a55510e2-3e8a-4b4d-865b-d41f30c9f726","Type":"ContainerDied","Data":"84beaab02f29e7bec784c021c61dcb15138409a5397aa4b9c0cdba30c35501db"}
Jul 27 20:21:23.801903 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.801714    2579 scope.go:117] "RemoveContainer" containerID="8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860"
Jul 27 20:21:23.803053 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.803028    2579 generic.go:356] "Generic (PLEG): container finished" podID="d505f05b-734c-4ee3-b8ca-13b1b174400d" containerID="5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0" exitCode=0
Jul 27 20:21:23.803185 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.803080    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"
Jul 27 20:21:23.803185 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.803151    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9" event={"ID":"d505f05b-734c-4ee3-b8ca-13b1b174400d","Type":"ContainerDied","Data":"5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0"}
Jul 27 20:21:23.803340 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.803192    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9" event={"ID":"d505f05b-734c-4ee3-b8ca-13b1b174400d","Type":"ContainerDied","Data":"a5359f54db5f0f3c46dca565c46ef5d603164ad9df24b571550701e2a53a6e8b"}
Jul 27 20:21:23.811878 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.811863    2579 scope.go:117] "RemoveContainer" containerID="8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860"
Jul 27 20:21:23.812165 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:21:23.812147    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860\": container with ID starting with 8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860 not found: ID does not exist" containerID="8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860"
Jul 27 20:21:23.812229 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.812173    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860"} err="failed to get container status \"8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860\": rpc error: code = NotFound desc = could not find container \"8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860\": container with ID starting with 8af8e7b239155333cc7c4dc454905bb4c690150c1bb2fe89dbf3987b11657860 not found: ID does not exist"
Jul 27 20:21:23.812229 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.812189    2579 scope.go:117] "RemoveContainer" containerID="5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0"
Jul 27 20:21:23.821800 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.821770    2579 scope.go:117] "RemoveContainer" containerID="5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0"
Jul 27 20:21:23.822105 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:21:23.822073    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0\": container with ID starting with 5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0 not found: ID does not exist" containerID="5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0"
Jul 27 20:21:23.822199 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.822109    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0"} err="failed to get container status \"5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0\": rpc error: code = NotFound desc = could not find container \"5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0\": container with ID starting with 5297e9d4ae3f9f8bb12e2b0c59c923e6afd6b7df4bd0bf30d28ec286653cedc0 not found: ID does not exist"
Jul 27 20:21:23.824064 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.824044    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"]
Jul 27 20:21:23.827823 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.827805    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6c472a-59cf5c48b8-hmqb9"]
Jul 27 20:21:23.837636 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.837615    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"]
Jul 27 20:21:23.840677 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:23.840655    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6c472a-5fcc4f657b-rpjd9"]
Jul 27 20:21:24.489467 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:24.489433    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a55510e2-3e8a-4b4d-865b-d41f30c9f726" path="/var/lib/kubelet/pods/a55510e2-3e8a-4b4d-865b-d41f30c9f726/volumes"
Jul 27 20:21:24.489851 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:24.489827    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d505f05b-734c-4ee3-b8ca-13b1b174400d" path="/var/lib/kubelet/pods/d505f05b-734c-4ee3-b8ca-13b1b174400d/volumes"
Jul 27 20:21:27.789306 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:27.789271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:28.179999 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:28.179947    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:33.156481 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:33.156444    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"]
Jul 27 20:21:33.202267 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:33.202204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:44.513582 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:44.513531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:21:48.179062 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.179015    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6" podUID="bc8baa2b-acd3-4eeb-a733-e943054d2ea5" containerName="main" containerID="cri-o://df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911" gracePeriod=45
Jul 27 20:21:48.438943 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.438880    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:21:48.520566 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520537    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-home\") pod \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") "
Jul 27 20:21:48.520749 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520648    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-dshm\") pod \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") "
Jul 27 20:21:48.520749 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520688    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-model-cache\") pod \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") "
Jul 27 20:21:48.520749 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520722    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tmp-dir\") pod \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") "
Jul 27 20:21:48.520923 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520769    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tls-certs\") pod \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") "
Jul 27 20:21:48.520923 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520791    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-home" (OuterVolumeSpecName: "home") pod "bc8baa2b-acd3-4eeb-a733-e943054d2ea5" (UID: "bc8baa2b-acd3-4eeb-a733-e943054d2ea5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:21:48.520923 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520805    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp2fx\" (UniqueName: \"kubernetes.io/projected/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-kube-api-access-dp2fx\") pod \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\" (UID: \"bc8baa2b-acd3-4eeb-a733-e943054d2ea5\") "
Jul 27 20:21:48.521068 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520917    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-model-cache" (OuterVolumeSpecName: "model-cache") pod "bc8baa2b-acd3-4eeb-a733-e943054d2ea5" (UID: "bc8baa2b-acd3-4eeb-a733-e943054d2ea5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:21:48.521068 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.520997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bc8baa2b-acd3-4eeb-a733-e943054d2ea5" (UID: "bc8baa2b-acd3-4eeb-a733-e943054d2ea5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:21:48.521190 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.521172    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tmp-dir\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:48.521263 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.521194    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-home\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:48.521263 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.521205    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-model-cache\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:48.522796 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.522775    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-dshm" (OuterVolumeSpecName: "dshm") pod "bc8baa2b-acd3-4eeb-a733-e943054d2ea5" (UID: "bc8baa2b-acd3-4eeb-a733-e943054d2ea5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:21:48.523213 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.523193    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bc8baa2b-acd3-4eeb-a733-e943054d2ea5" (UID: "bc8baa2b-acd3-4eeb-a733-e943054d2ea5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:21:48.523289 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.523224    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-kube-api-access-dp2fx" (OuterVolumeSpecName: "kube-api-access-dp2fx") pod "bc8baa2b-acd3-4eeb-a733-e943054d2ea5" (UID: "bc8baa2b-acd3-4eeb-a733-e943054d2ea5"). InnerVolumeSpecName "kube-api-access-dp2fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:21:48.621854 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.621827    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-dshm\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:48.621854 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.621850    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-tls-certs\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:48.622039 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.621861    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp2fx\" (UniqueName: \"kubernetes.io/projected/bc8baa2b-acd3-4eeb-a733-e943054d2ea5-kube-api-access-dp2fx\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:21:48.937322 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.937284    2579 generic.go:356] "Generic (PLEG): container finished" podID="bc8baa2b-acd3-4eeb-a733-e943054d2ea5" containerID="df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911" exitCode=0
Jul 27 20:21:48.937461 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.937331    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"
Jul 27 20:21:48.937461 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.937370    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6" event={"ID":"bc8baa2b-acd3-4eeb-a733-e943054d2ea5","Type":"ContainerDied","Data":"df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911"}
Jul 27 20:21:48.937461 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.937417    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6" event={"ID":"bc8baa2b-acd3-4eeb-a733-e943054d2ea5","Type":"ContainerDied","Data":"9b28573f85a214291c1c70db6152d4070d2c853670e7e6d4fa9a6eabbfdf4961"}
Jul 27 20:21:48.937461 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.937442    2579 scope.go:117] "RemoveContainer" containerID="df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911"
Jul 27 20:21:48.946349 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.946323    2579 scope.go:117] "RemoveContainer" containerID="df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911"
Jul 27 20:21:48.946597 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:21:48.946576    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911\": container with ID starting with df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911 not found: ID does not exist" containerID="df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911"
Jul 27 20:21:48.946640 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.946606    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911"} err="failed to get container status \"df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911\": rpc error: code = NotFound desc = could not find container \"df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911\": container with ID starting with df9c7b2be467ec11f155b985702856247ed09c196f27d9cabf132ec23c266911 not found: ID does not exist"
Jul 27 20:21:48.958932 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.958903    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"]
Jul 27 20:21:48.961739 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:48.961720    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-6c472a/ipp-route-6c472a-kserve-7fcb968f79-b26b6"]
Jul 27 20:21:50.488992 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:21:50.488958    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc8baa2b-acd3-4eeb-a733-e943054d2ea5" path="/var/lib/kubelet/pods/bc8baa2b-acd3-4eeb-a733-e943054d2ea5/volumes"
Jul 27 20:22:09.984718 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:09.984675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:10.317235 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:10.317154    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:15.843122 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:15.843088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:15.943910 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:15.943877    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:16.106230 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.106135    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:16.338742 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.338708    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:16.460878 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.460835    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"]
Jul 27 20:22:16.461371 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461352    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d505f05b-734c-4ee3-b8ca-13b1b174400d" containerName="payload-pre-processing"
Jul 27 20:22:16.461496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461372    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d505f05b-734c-4ee3-b8ca-13b1b174400d" containerName="payload-pre-processing"
Jul 27 20:22:16.461496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461407    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a55510e2-3e8a-4b4d-865b-d41f30c9f726" containerName="payload-processing"
Jul 27 20:22:16.461496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461416    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a55510e2-3e8a-4b4d-865b-d41f30c9f726" containerName="payload-processing"
Jul 27 20:22:16.461496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461427    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8baa2b-acd3-4eeb-a733-e943054d2ea5" containerName="main"
Jul 27 20:22:16.461496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461439    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8baa2b-acd3-4eeb-a733-e943054d2ea5" containerName="main"
Jul 27 20:22:16.461496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461452    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad01d03c-4cde-498c-88b1-fe000c8a2ca0" containerName="payload-processing"
Jul 27 20:22:16.461496 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461460    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad01d03c-4cde-498c-88b1-fe000c8a2ca0" containerName="payload-processing"
Jul 27 20:22:16.461846 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461571    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad01d03c-4cde-498c-88b1-fe000c8a2ca0" containerName="payload-processing"
Jul 27 20:22:16.461846 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461585    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a55510e2-3e8a-4b4d-865b-d41f30c9f726" containerName="payload-processing"
Jul 27 20:22:16.461846 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461598    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc8baa2b-acd3-4eeb-a733-e943054d2ea5" containerName="main"
Jul 27 20:22:16.461846 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.461609    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d505f05b-734c-4ee3-b8ca-13b1b174400d" containerName="payload-pre-processing"
Jul 27 20:22:16.465060 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.465039    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"
Jul 27 20:22:16.467494 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.467452    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 20:22:16.471686 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.471665    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"]
Jul 27 20:22:16.594780 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.594743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b09845a-dc35-4203-a21b-3ad278617e1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh\" (UID: \"3b09845a-dc35-4203-a21b-3ad278617e1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"
Jul 27 20:22:16.695361 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.695323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b09845a-dc35-4203-a21b-3ad278617e1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh\" (UID: \"3b09845a-dc35-4203-a21b-3ad278617e1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"
Jul 27 20:22:16.695954 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.695936    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b09845a-dc35-4203-a21b-3ad278617e1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh\" (UID: \"3b09845a-dc35-4203-a21b-3ad278617e1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"
Jul 27 20:22:16.778659 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.778496    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"
Jul 27 20:22:16.911548 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:16.911518    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"]
Jul 27 20:22:16.913819 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:22:16.913779    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b09845a_dc35_4203_a21b_3ad278617e1e.slice/crio-cf6187cd9f1d267dd2a2f7b2daa26a02e1b99b98e4860008cf9b2849f3d5259c WatchSource:0}: Error finding container cf6187cd9f1d267dd2a2f7b2daa26a02e1b99b98e4860008cf9b2849f3d5259c: Status 404 returned error can't find the container with id cf6187cd9f1d267dd2a2f7b2daa26a02e1b99b98e4860008cf9b2849f3d5259c
Jul 27 20:22:17.083486 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:17.083397    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh" event={"ID":"3b09845a-dc35-4203-a21b-3ad278617e1e","Type":"ContainerStarted","Data":"cf6187cd9f1d267dd2a2f7b2daa26a02e1b99b98e4860008cf9b2849f3d5259c"}
Jul 27 20:22:18.089938 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:18.089847    2579 generic.go:356] "Generic (PLEG): container finished" podID="3b09845a-dc35-4203-a21b-3ad278617e1e" containerID="0598c3760acb30ce233384d9197209a5f517d853d5bae532590b2ff14217ffc4" exitCode=0
Jul 27 20:22:18.089938 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:18.089926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh" event={"ID":"3b09845a-dc35-4203-a21b-3ad278617e1e","Type":"ContainerDied","Data":"0598c3760acb30ce233384d9197209a5f517d853d5bae532590b2ff14217ffc4"}
Jul 27 20:22:19.237975 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:19.237946    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"
Jul 27 20:22:19.422151 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:19.422064    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b09845a-dc35-4203-a21b-3ad278617e1e-maas-api-service-ca\") pod \"3b09845a-dc35-4203-a21b-3ad278617e1e\" (UID: \"3b09845a-dc35-4203-a21b-3ad278617e1e\") "
Jul 27 20:22:19.422514 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:19.422493    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b09845a-dc35-4203-a21b-3ad278617e1e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b09845a-dc35-4203-a21b-3ad278617e1e" (UID: "3b09845a-dc35-4203-a21b-3ad278617e1e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:22:19.522960 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:19.522931    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b09845a-dc35-4203-a21b-3ad278617e1e-maas-api-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:20.102203 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:20.102175    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"
Jul 27 20:22:20.102203 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:20.102197    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh" event={"ID":"3b09845a-dc35-4203-a21b-3ad278617e1e","Type":"ContainerDied","Data":"cf6187cd9f1d267dd2a2f7b2daa26a02e1b99b98e4860008cf9b2849f3d5259c"}
Jul 27 20:22:20.102434 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:20.102264    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf6187cd9f1d267dd2a2f7b2daa26a02e1b99b98e4860008cf9b2849f3d5259c"
Jul 27 20:22:26.870618 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:26.870579    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:26.916262 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:26.916219    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"]
Jul 27 20:22:26.916528 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:26.916499    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m" podUID="611cc43a-8fd7-438a-a098-efef6f938347" containerName="payload-processing" containerID="cri-o://4a11c1609ad54bb2b616723f9718f2676785569bb647cba33edd17b35fa9ae0e" gracePeriod=30
Jul 27 20:22:26.919619 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:26.919592    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"]
Jul 27 20:22:26.919931 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:26.919882    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l" podUID="6be7f991-1cee-4d32-9fa7-f83020eb343c" containerName="payload-pre-processing" containerID="cri-o://9fe8417215f5041bb0231a3434aefa5c8a283a738e253ddbdd6102478b5f6190" gracePeriod=30
Jul 27 20:22:26.941738 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:26.941706    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:26.998183 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:26.998157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:27.143384 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.143352    2579 generic.go:356] "Generic (PLEG): container finished" podID="6be7f991-1cee-4d32-9fa7-f83020eb343c" containerID="9fe8417215f5041bb0231a3434aefa5c8a283a738e253ddbdd6102478b5f6190" exitCode=0
Jul 27 20:22:27.143572 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.143431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l" event={"ID":"6be7f991-1cee-4d32-9fa7-f83020eb343c","Type":"ContainerDied","Data":"9fe8417215f5041bb0231a3434aefa5c8a283a738e253ddbdd6102478b5f6190"}
Jul 27 20:22:27.145315 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.145293    2579 generic.go:356] "Generic (PLEG): container finished" podID="611cc43a-8fd7-438a-a098-efef6f938347" containerID="4a11c1609ad54bb2b616723f9718f2676785569bb647cba33edd17b35fa9ae0e" exitCode=0
Jul 27 20:22:27.145456 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.145355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m" event={"ID":"611cc43a-8fd7-438a-a098-efef6f938347","Type":"ContainerDied","Data":"4a11c1609ad54bb2b616723f9718f2676785569bb647cba33edd17b35fa9ae0e"}
Jul 27 20:22:27.200443 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.200418    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:22:27.203386 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.203368    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:22:27.293684 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.293649    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4rts\" (UniqueName: \"kubernetes.io/projected/611cc43a-8fd7-438a-a098-efef6f938347-kube-api-access-g4rts\") pod \"611cc43a-8fd7-438a-a098-efef6f938347\" (UID: \"611cc43a-8fd7-438a-a098-efef6f938347\") "
Jul 27 20:22:27.293876 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.293734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znd7d\" (UniqueName: \"kubernetes.io/projected/6be7f991-1cee-4d32-9fa7-f83020eb343c-kube-api-access-znd7d\") pod \"6be7f991-1cee-4d32-9fa7-f83020eb343c\" (UID: \"6be7f991-1cee-4d32-9fa7-f83020eb343c\") "
Jul 27 20:22:27.293876 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.293770    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/611cc43a-8fd7-438a-a098-efef6f938347-plugins-config-volume\") pod \"611cc43a-8fd7-438a-a098-efef6f938347\" (UID: \"611cc43a-8fd7-438a-a098-efef6f938347\") "
Jul 27 20:22:27.293876 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.293828    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be7f991-1cee-4d32-9fa7-f83020eb343c-plugins-config-volume\") pod \"6be7f991-1cee-4d32-9fa7-f83020eb343c\" (UID: \"6be7f991-1cee-4d32-9fa7-f83020eb343c\") "
Jul 27 20:22:27.294201 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.294170    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/611cc43a-8fd7-438a-a098-efef6f938347-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "611cc43a-8fd7-438a-a098-efef6f938347" (UID: "611cc43a-8fd7-438a-a098-efef6f938347"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:22:27.294334 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.294227    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be7f991-1cee-4d32-9fa7-f83020eb343c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6be7f991-1cee-4d32-9fa7-f83020eb343c" (UID: "6be7f991-1cee-4d32-9fa7-f83020eb343c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:22:27.295683 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.295659    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611cc43a-8fd7-438a-a098-efef6f938347-kube-api-access-g4rts" (OuterVolumeSpecName: "kube-api-access-g4rts") pod "611cc43a-8fd7-438a-a098-efef6f938347" (UID: "611cc43a-8fd7-438a-a098-efef6f938347"). InnerVolumeSpecName "kube-api-access-g4rts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:22:27.295799 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.295754    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be7f991-1cee-4d32-9fa7-f83020eb343c-kube-api-access-znd7d" (OuterVolumeSpecName: "kube-api-access-znd7d") pod "6be7f991-1cee-4d32-9fa7-f83020eb343c" (UID: "6be7f991-1cee-4d32-9fa7-f83020eb343c"). InnerVolumeSpecName "kube-api-access-znd7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:22:27.394826 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.394738    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znd7d\" (UniqueName: \"kubernetes.io/projected/6be7f991-1cee-4d32-9fa7-f83020eb343c-kube-api-access-znd7d\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:27.394826 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.394770    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/611cc43a-8fd7-438a-a098-efef6f938347-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:27.394826 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.394780    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6be7f991-1cee-4d32-9fa7-f83020eb343c-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:27.394826 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:27.394789    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4rts\" (UniqueName: \"kubernetes.io/projected/611cc43a-8fd7-438a-a098-efef6f938347-kube-api-access-g4rts\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:28.152190 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.152160    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"
Jul 27 20:22:28.152660 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.152156    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l" event={"ID":"6be7f991-1cee-4d32-9fa7-f83020eb343c","Type":"ContainerDied","Data":"30ed0462c81a6156365178b22d7427b4246c2058475fabaab26a6ece52d01481"}
Jul 27 20:22:28.152660 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.152326    2579 scope.go:117] "RemoveContainer" containerID="9fe8417215f5041bb0231a3434aefa5c8a283a738e253ddbdd6102478b5f6190"
Jul 27 20:22:28.161442 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.161322    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m" event={"ID":"611cc43a-8fd7-438a-a098-efef6f938347","Type":"ContainerDied","Data":"0ec8c986c7b454b0a33dfde8ab5f69a25495ae942562de486b70e6972d7621f9"}
Jul 27 20:22:28.161569 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.161460    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"
Jul 27 20:22:28.169904 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.169876    2579 scope.go:117] "RemoveContainer" containerID="4a11c1609ad54bb2b616723f9718f2676785569bb647cba33edd17b35fa9ae0e"
Jul 27 20:22:28.179467 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.179447    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"]
Jul 27 20:22:28.183416 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.183394    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d4162b-5d86df7c6-9f74l"]
Jul 27 20:22:28.192973 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.192951    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"]
Jul 27 20:22:28.195787 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.195768    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d4162b-54686ff5b6-dcq4m"]
Jul 27 20:22:28.488819 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.488785    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="611cc43a-8fd7-438a-a098-efef6f938347" path="/var/lib/kubelet/pods/611cc43a-8fd7-438a-a098-efef6f938347/volumes"
Jul 27 20:22:28.489259 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:28.489227    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6be7f991-1cee-4d32-9fa7-f83020eb343c" path="/var/lib/kubelet/pods/6be7f991-1cee-4d32-9fa7-f83020eb343c/volumes"
Jul 27 20:22:31.527381 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:31.527329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:31.532233 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:31.532208    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"]
Jul 27 20:22:31.534065 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:31.534041    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d4162b-4bfgh"]
Jul 27 20:22:31.923798 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:31.923705    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:32.489299 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:32.489266    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b09845a-dc35-4203-a21b-3ad278617e1e" path="/var/lib/kubelet/pods/3b09845a-dc35-4203-a21b-3ad278617e1e/volumes"
Jul 27 20:22:37.788844 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:37.788813    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:37.845573 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:37.845540    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:38.052694 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.052610    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:38.286220 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.286181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:38.427835 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.427807    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"]
Jul 27 20:22:38.428214 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428202    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be7f991-1cee-4d32-9fa7-f83020eb343c" containerName="payload-pre-processing"
Jul 27 20:22:38.428267 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428216    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be7f991-1cee-4d32-9fa7-f83020eb343c" containerName="payload-pre-processing"
Jul 27 20:22:38.428267 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428235    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b09845a-dc35-4203-a21b-3ad278617e1e" containerName="revoke-keys"
Jul 27 20:22:38.428267 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428252    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b09845a-dc35-4203-a21b-3ad278617e1e" containerName="revoke-keys"
Jul 27 20:22:38.428367 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428268    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611cc43a-8fd7-438a-a098-efef6f938347" containerName="payload-processing"
Jul 27 20:22:38.428367 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428274    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="611cc43a-8fd7-438a-a098-efef6f938347" containerName="payload-processing"
Jul 27 20:22:38.428367 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428334    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be7f991-1cee-4d32-9fa7-f83020eb343c" containerName="payload-pre-processing"
Jul 27 20:22:38.428367 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428342    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b09845a-dc35-4203-a21b-3ad278617e1e" containerName="revoke-keys"
Jul 27 20:22:38.428367 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.428349    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="611cc43a-8fd7-438a-a098-efef6f938347" containerName="payload-processing"
Jul 27 20:22:38.431500 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.431484    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"
Jul 27 20:22:38.434156 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.434135    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 20:22:38.438982 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.438960    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"]
Jul 27 20:22:38.500267 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.500218    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebae7d1-4263-44f7-8de9-0b50fbabac60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg\" (UID: \"eebae7d1-4263-44f7-8de9-0b50fbabac60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"
Jul 27 20:22:38.601029 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.600991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebae7d1-4263-44f7-8de9-0b50fbabac60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg\" (UID: \"eebae7d1-4263-44f7-8de9-0b50fbabac60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"
Jul 27 20:22:38.601693 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.601675    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebae7d1-4263-44f7-8de9-0b50fbabac60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg\" (UID: \"eebae7d1-4263-44f7-8de9-0b50fbabac60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"
Jul 27 20:22:38.744162 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.744065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"
Jul 27 20:22:38.876016 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:38.875971    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"]
Jul 27 20:22:38.878965 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:22:38.878938    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebae7d1_4263_44f7_8de9_0b50fbabac60.slice/crio-a5071cd0726c03f4d0fa9ebd9c221f8775f7c0be0d43f00f6228834c64f26fa7 WatchSource:0}: Error finding container a5071cd0726c03f4d0fa9ebd9c221f8775f7c0be0d43f00f6228834c64f26fa7: Status 404 returned error can't find the container with id a5071cd0726c03f4d0fa9ebd9c221f8775f7c0be0d43f00f6228834c64f26fa7
Jul 27 20:22:39.228422 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:39.228378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg" event={"ID":"eebae7d1-4263-44f7-8de9-0b50fbabac60","Type":"ContainerStarted","Data":"5b3313e1a466cf9e7a22081f7e66b7ae88cf72e88928a1e4c09f4504a558e6a2"}
Jul 27 20:22:39.228581 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:39.228440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg" event={"ID":"eebae7d1-4263-44f7-8de9-0b50fbabac60","Type":"ContainerStarted","Data":"a5071cd0726c03f4d0fa9ebd9c221f8775f7c0be0d43f00f6228834c64f26fa7"}
Jul 27 20:22:39.243081 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:39.243034    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg" podStartSLOduration=1.243005575 podStartE2EDuration="1.243005575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:22:39.242747987 +0000 UTC m=+3411.334099486" watchObservedRunningTime="2026-07-27 20:22:39.243005575 +0000 UTC m=+3411.334357073"
Jul 27 20:22:40.235442 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:40.235409    2579 generic.go:356] "Generic (PLEG): container finished" podID="eebae7d1-4263-44f7-8de9-0b50fbabac60" containerID="5b3313e1a466cf9e7a22081f7e66b7ae88cf72e88928a1e4c09f4504a558e6a2" exitCode=0
Jul 27 20:22:40.235887 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:40.235492    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg" event={"ID":"eebae7d1-4263-44f7-8de9-0b50fbabac60","Type":"ContainerDied","Data":"5b3313e1a466cf9e7a22081f7e66b7ae88cf72e88928a1e4c09f4504a558e6a2"}
Jul 27 20:22:41.381785 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:41.381763    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"
Jul 27 20:22:41.532636 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:41.532564    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebae7d1-4263-44f7-8de9-0b50fbabac60-maas-api-service-ca\") pod \"eebae7d1-4263-44f7-8de9-0b50fbabac60\" (UID: \"eebae7d1-4263-44f7-8de9-0b50fbabac60\") "
Jul 27 20:22:41.532896 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:41.532874    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eebae7d1-4263-44f7-8de9-0b50fbabac60-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eebae7d1-4263-44f7-8de9-0b50fbabac60" (UID: "eebae7d1-4263-44f7-8de9-0b50fbabac60"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:22:41.633494 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:41.633462    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebae7d1-4263-44f7-8de9-0b50fbabac60-maas-api-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:42.247829 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:42.247800    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"
Jul 27 20:22:42.247829 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:42.247821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg" event={"ID":"eebae7d1-4263-44f7-8de9-0b50fbabac60","Type":"ContainerDied","Data":"a5071cd0726c03f4d0fa9ebd9c221f8775f7c0be0d43f00f6228834c64f26fa7"}
Jul 27 20:22:42.248055 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:42.247873    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5071cd0726c03f4d0fa9ebd9c221f8775f7c0be0d43f00f6228834c64f26fa7"
Jul 27 20:22:48.843440 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.843399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:48.847664 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.847633    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"]
Jul 27 20:22:48.849586 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.849568    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-97e28d-29753055-gdrnb"]
Jul 27 20:22:48.873091 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.873064    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"]
Jul 27 20:22:48.873315 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.873284    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh" podUID="0150142d-7c7a-4413-acd3-b3b196107d58" containerName="payload-processing" containerID="cri-o://63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0" gracePeriod=30
Jul 27 20:22:48.880573 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.880538    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"]
Jul 27 20:22:48.881037 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.880965    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz" podUID="13ff142c-a911-46bf-b471-7825d327add4" containerName="payload-pre-processing" containerID="cri-o://64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c" gracePeriod=30
Jul 27 20:22:48.913934 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.913901    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:48.971445 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:48.971412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:49.162064 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.162041    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:22:49.164042 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.164021    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:22:49.287761 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.287726    2579 generic.go:356] "Generic (PLEG): container finished" podID="0150142d-7c7a-4413-acd3-b3b196107d58" containerID="63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0" exitCode=0
Jul 27 20:22:49.287940 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.287774    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"
Jul 27 20:22:49.287940 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.287827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh" event={"ID":"0150142d-7c7a-4413-acd3-b3b196107d58","Type":"ContainerDied","Data":"63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0"}
Jul 27 20:22:49.287940 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.287892    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh" event={"ID":"0150142d-7c7a-4413-acd3-b3b196107d58","Type":"ContainerDied","Data":"6209baef6b36c314f8d5ffdba7e5444a3ba9759137ccc0234d265a2fb04d368e"}
Jul 27 20:22:49.287940 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.287925    2579 scope.go:117] "RemoveContainer" containerID="63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0"
Jul 27 20:22:49.289061 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.289040    2579 generic.go:356] "Generic (PLEG): container finished" podID="13ff142c-a911-46bf-b471-7825d327add4" containerID="64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c" exitCode=0
Jul 27 20:22:49.289159 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.289065    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz" event={"ID":"13ff142c-a911-46bf-b471-7825d327add4","Type":"ContainerDied","Data":"64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c"}
Jul 27 20:22:49.289159 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.289103    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"
Jul 27 20:22:49.289159 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.289130    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz" event={"ID":"13ff142c-a911-46bf-b471-7825d327add4","Type":"ContainerDied","Data":"9ab6c04e890367c7a1eb3f41b934034a9571d301ce739e9f7960afb9f7d5da69"}
Jul 27 20:22:49.297521 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.297507    2579 scope.go:117] "RemoveContainer" containerID="63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0"
Jul 27 20:22:49.297760 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:22:49.297734    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0\": container with ID starting with 63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0 not found: ID does not exist" containerID="63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0"
Jul 27 20:22:49.297869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.297761    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0"} err="failed to get container status \"63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0\": rpc error: code = NotFound desc = could not find container \"63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0\": container with ID starting with 63840d0ac7f48e54fea3fc0b48aad7ae5ef1c0263d2ce271c0953079d6660ad0 not found: ID does not exist"
Jul 27 20:22:49.297869 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.297782    2579 scope.go:117] "RemoveContainer" containerID="64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c"
Jul 27 20:22:49.308143 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.308121    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0150142d-7c7a-4413-acd3-b3b196107d58-plugins-config-volume\") pod \"0150142d-7c7a-4413-acd3-b3b196107d58\" (UID: \"0150142d-7c7a-4413-acd3-b3b196107d58\") "
Jul 27 20:22:49.308287 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.308274    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n94pv\" (UniqueName: \"kubernetes.io/projected/0150142d-7c7a-4413-acd3-b3b196107d58-kube-api-access-n94pv\") pod \"0150142d-7c7a-4413-acd3-b3b196107d58\" (UID: \"0150142d-7c7a-4413-acd3-b3b196107d58\") "
Jul 27 20:22:49.308369 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.308301    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ff142c-a911-46bf-b471-7825d327add4-plugins-config-volume\") pod \"13ff142c-a911-46bf-b471-7825d327add4\" (UID: \"13ff142c-a911-46bf-b471-7825d327add4\") "
Jul 27 20:22:49.308432 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.308388    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8mrt\" (UniqueName: \"kubernetes.io/projected/13ff142c-a911-46bf-b471-7825d327add4-kube-api-access-g8mrt\") pod \"13ff142c-a911-46bf-b471-7825d327add4\" (UID: \"13ff142c-a911-46bf-b471-7825d327add4\") "
Jul 27 20:22:49.308585 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.308559    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0150142d-7c7a-4413-acd3-b3b196107d58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0150142d-7c7a-4413-acd3-b3b196107d58" (UID: "0150142d-7c7a-4413-acd3-b3b196107d58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:22:49.308652 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.308632    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ff142c-a911-46bf-b471-7825d327add4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13ff142c-a911-46bf-b471-7825d327add4" (UID: "13ff142c-a911-46bf-b471-7825d327add4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:22:49.308775 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.308756    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ff142c-a911-46bf-b471-7825d327add4-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:49.308841 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.308781    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0150142d-7c7a-4413-acd3-b3b196107d58-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:49.310645 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.310617    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0150142d-7c7a-4413-acd3-b3b196107d58-kube-api-access-n94pv" (OuterVolumeSpecName: "kube-api-access-n94pv") pod "0150142d-7c7a-4413-acd3-b3b196107d58" (UID: "0150142d-7c7a-4413-acd3-b3b196107d58"). InnerVolumeSpecName "kube-api-access-n94pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:22:49.310747 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.310655    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ff142c-a911-46bf-b471-7825d327add4-kube-api-access-g8mrt" (OuterVolumeSpecName: "kube-api-access-g8mrt") pod "13ff142c-a911-46bf-b471-7825d327add4" (UID: "13ff142c-a911-46bf-b471-7825d327add4"). InnerVolumeSpecName "kube-api-access-g8mrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:22:49.311337 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.311318    2579 scope.go:117] "RemoveContainer" containerID="64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c"
Jul 27 20:22:49.311589 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:22:49.311567    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c\": container with ID starting with 64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c not found: ID does not exist" containerID="64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c"
Jul 27 20:22:49.311691 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.311593    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c"} err="failed to get container status \"64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c\": rpc error: code = NotFound desc = could not find container \"64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c\": container with ID starting with 64df8dc20af6a290b5252606583ec7cc4a94b3aae00308ad22f6765b708b7f1c not found: ID does not exist"
Jul 27 20:22:49.409309 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.409206    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n94pv\" (UniqueName: \"kubernetes.io/projected/0150142d-7c7a-4413-acd3-b3b196107d58-kube-api-access-n94pv\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:49.409309 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.409234    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8mrt\" (UniqueName: \"kubernetes.io/projected/13ff142c-a911-46bf-b471-7825d327add4-kube-api-access-g8mrt\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:22:49.612633 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.612601    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"]
Jul 27 20:22:49.616145 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.616115    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-97e28d-6b745d98d6-vs9gh"]
Jul 27 20:22:49.625177 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.625154    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"]
Jul 27 20:22:49.628432 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:49.628408    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-97e28d-78bdccbd-kx5zz"]
Jul 27 20:22:50.488978 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:50.488945    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0150142d-7c7a-4413-acd3-b3b196107d58" path="/var/lib/kubelet/pods/0150142d-7c7a-4413-acd3-b3b196107d58/volumes"
Jul 27 20:22:50.489381 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:50.489361    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091ac799-85a2-4746-be05-2935b696afa8" path="/var/lib/kubelet/pods/091ac799-85a2-4746-be05-2935b696afa8/volumes"
Jul 27 20:22:50.489655 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:50.489641    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13ff142c-a911-46bf-b471-7825d327add4" path="/var/lib/kubelet/pods/13ff142c-a911-46bf-b471-7825d327add4/volumes"
Jul 27 20:22:53.495136 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:53.495079    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"]
Jul 27 20:22:53.497579 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:53.497555    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-97e28d-bkzkg"]
Jul 27 20:22:53.514133 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:53.514104    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:53.636807 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:53.636763    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:53.712646 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:53.712618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:53.871361 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:53.871225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:54.488941 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:54.488908    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eebae7d1-4263-44f7-8de9-0b50fbabac60" path="/var/lib/kubelet/pods/eebae7d1-4263-44f7-8de9-0b50fbabac60/volumes"
Jul 27 20:22:59.781608 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:59.781579    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:22:59.842149 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:22:59.842116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:00.067669 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.067568    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:00.316427 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.316392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:00.414667 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.414586    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"]
Jul 27 20:23:00.414990 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.414978    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0150142d-7c7a-4413-acd3-b3b196107d58" containerName="payload-processing"
Jul 27 20:23:00.415039 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.414993    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0150142d-7c7a-4413-acd3-b3b196107d58" containerName="payload-processing"
Jul 27 20:23:00.415039 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.415008    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eebae7d1-4263-44f7-8de9-0b50fbabac60" containerName="revoke-keys"
Jul 27 20:23:00.415039 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.415014    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eebae7d1-4263-44f7-8de9-0b50fbabac60" containerName="revoke-keys"
Jul 27 20:23:00.415039 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.415021    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ff142c-a911-46bf-b471-7825d327add4" containerName="payload-pre-processing"
Jul 27 20:23:00.415039 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.415028    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ff142c-a911-46bf-b471-7825d327add4" containerName="payload-pre-processing"
Jul 27 20:23:00.415194 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.415092    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0150142d-7c7a-4413-acd3-b3b196107d58" containerName="payload-processing"
Jul 27 20:23:00.415194 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.415100    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ff142c-a911-46bf-b471-7825d327add4" containerName="payload-pre-processing"
Jul 27 20:23:00.415194 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.415110    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eebae7d1-4263-44f7-8de9-0b50fbabac60" containerName="revoke-keys"
Jul 27 20:23:00.418177 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.418157    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"
Jul 27 20:23:00.420502 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.420483    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-86v9x\""
Jul 27 20:23:00.425562 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.425533    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"]
Jul 27 20:23:00.512336 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.512302    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6823ec-5269-4b12-885d-8654ac2b9be0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj\" (UID: \"6c6823ec-5269-4b12-885d-8654ac2b9be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"
Jul 27 20:23:00.613186 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.613155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6823ec-5269-4b12-885d-8654ac2b9be0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj\" (UID: \"6c6823ec-5269-4b12-885d-8654ac2b9be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"
Jul 27 20:23:00.613788 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.613770    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6823ec-5269-4b12-885d-8654ac2b9be0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj\" (UID: \"6c6823ec-5269-4b12-885d-8654ac2b9be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"
Jul 27 20:23:00.730516 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.730478    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"
Jul 27 20:23:00.859520 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:00.859489    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"]
Jul 27 20:23:00.861306 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:23:00.861280    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6823ec_5269_4b12_885d_8654ac2b9be0.slice/crio-50f5b5351c1d9b60d7eaf634dc4031b3d7d206231a8bf8b0fc4788e1cdcd00bc WatchSource:0}: Error finding container 50f5b5351c1d9b60d7eaf634dc4031b3d7d206231a8bf8b0fc4788e1cdcd00bc: Status 404 returned error can't find the container with id 50f5b5351c1d9b60d7eaf634dc4031b3d7d206231a8bf8b0fc4788e1cdcd00bc
Jul 27 20:23:01.360196 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:01.360100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj" event={"ID":"6c6823ec-5269-4b12-885d-8654ac2b9be0","Type":"ContainerStarted","Data":"ef91dcec2709a9d90b436d41fe7cb7b6e55bf509b4ac308dd3631fc3eb0a2f1f"}
Jul 27 20:23:01.360196 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:01.360192    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj" event={"ID":"6c6823ec-5269-4b12-885d-8654ac2b9be0","Type":"ContainerStarted","Data":"50f5b5351c1d9b60d7eaf634dc4031b3d7d206231a8bf8b0fc4788e1cdcd00bc"}
Jul 27 20:23:01.374822 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:01.374775    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj" podStartSLOduration=1.374760628 podStartE2EDuration="1.374760628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:23:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:23:01.373883308 +0000 UTC m=+3433.465234807" watchObservedRunningTime="2026-07-27 20:23:01.374760628 +0000 UTC m=+3433.466112125"
Jul 27 20:23:02.366280 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:02.366232    2579 generic.go:356] "Generic (PLEG): container finished" podID="6c6823ec-5269-4b12-885d-8654ac2b9be0" containerID="ef91dcec2709a9d90b436d41fe7cb7b6e55bf509b4ac308dd3631fc3eb0a2f1f" exitCode=0
Jul 27 20:23:02.366665 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:02.366318    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj" event={"ID":"6c6823ec-5269-4b12-885d-8654ac2b9be0","Type":"ContainerDied","Data":"ef91dcec2709a9d90b436d41fe7cb7b6e55bf509b4ac308dd3631fc3eb0a2f1f"}
Jul 27 20:23:03.522961 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:03.522938    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"
Jul 27 20:23:03.541668 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:03.541638    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6823ec-5269-4b12-885d-8654ac2b9be0-maas-api-service-ca\") pod \"6c6823ec-5269-4b12-885d-8654ac2b9be0\" (UID: \"6c6823ec-5269-4b12-885d-8654ac2b9be0\") "
Jul 27 20:23:03.541950 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:03.541930    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c6823ec-5269-4b12-885d-8654ac2b9be0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6c6823ec-5269-4b12-885d-8654ac2b9be0" (UID: "6c6823ec-5269-4b12-885d-8654ac2b9be0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:23:03.542035 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:03.542023    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6823ec-5269-4b12-885d-8654ac2b9be0-maas-api-service-ca\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:23:04.377756 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:04.377725    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"
Jul 27 20:23:04.377937 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:04.377761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj" event={"ID":"6c6823ec-5269-4b12-885d-8654ac2b9be0","Type":"ContainerDied","Data":"50f5b5351c1d9b60d7eaf634dc4031b3d7d206231a8bf8b0fc4788e1cdcd00bc"}
Jul 27 20:23:04.377937 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:04.377807    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50f5b5351c1d9b60d7eaf634dc4031b3d7d206231a8bf8b0fc4788e1cdcd00bc"
Jul 27 20:23:10.842789 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:10.842752    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:10.862633 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:10.862603    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"]
Jul 27 20:23:10.862887 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:10.862859    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw" podUID="26539b65-f182-444b-b41f-ccd923c9b815" containerName="payload-pre-processing" containerID="cri-o://2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf" gracePeriod=30
Jul 27 20:23:10.943477 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:10.943446    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:11.120880 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.120857    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:23:11.215263 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.215201    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26539b65-f182-444b-b41f-ccd923c9b815-plugins-config-volume\") pod \"26539b65-f182-444b-b41f-ccd923c9b815\" (UID: \"26539b65-f182-444b-b41f-ccd923c9b815\") "
Jul 27 20:23:11.215263 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.215262    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp895\" (UniqueName: \"kubernetes.io/projected/26539b65-f182-444b-b41f-ccd923c9b815-kube-api-access-kp895\") pod \"26539b65-f182-444b-b41f-ccd923c9b815\" (UID: \"26539b65-f182-444b-b41f-ccd923c9b815\") "
Jul 27 20:23:11.215599 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.215575    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26539b65-f182-444b-b41f-ccd923c9b815-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26539b65-f182-444b-b41f-ccd923c9b815" (UID: "26539b65-f182-444b-b41f-ccd923c9b815"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:23:11.217388 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.217362    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26539b65-f182-444b-b41f-ccd923c9b815-kube-api-access-kp895" (OuterVolumeSpecName: "kube-api-access-kp895") pod "26539b65-f182-444b-b41f-ccd923c9b815" (UID: "26539b65-f182-444b-b41f-ccd923c9b815"). InnerVolumeSpecName "kube-api-access-kp895". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:23:11.316234 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.316205    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26539b65-f182-444b-b41f-ccd923c9b815-plugins-config-volume\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:23:11.316234 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.316232    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp895\" (UniqueName: \"kubernetes.io/projected/26539b65-f182-444b-b41f-ccd923c9b815-kube-api-access-kp895\") on node \"ip-10-0-131-59.ec2.internal\" DevicePath \"\""
Jul 27 20:23:11.414214 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.414122    2579 generic.go:356] "Generic (PLEG): container finished" podID="26539b65-f182-444b-b41f-ccd923c9b815" containerID="2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf" exitCode=0
Jul 27 20:23:11.414214 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.414167    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"
Jul 27 20:23:11.414412 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.414205    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw" event={"ID":"26539b65-f182-444b-b41f-ccd923c9b815","Type":"ContainerDied","Data":"2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf"}
Jul 27 20:23:11.414412 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.414282    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw" event={"ID":"26539b65-f182-444b-b41f-ccd923c9b815","Type":"ContainerDied","Data":"a6e15b99361519cc364250616eb1f61966ef0fb548882e9740b963bd115dd1ce"}
Jul 27 20:23:11.414412 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.414312    2579 scope.go:117] "RemoveContainer" containerID="2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf"
Jul 27 20:23:11.423285 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.423226    2579 scope.go:117] "RemoveContainer" containerID="2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf"
Jul 27 20:23:11.423539 ip-10-0-131-59 kubenswrapper[2579]: E0727 20:23:11.423520    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf\": container with ID starting with 2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf not found: ID does not exist" containerID="2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf"
Jul 27 20:23:11.423595 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.423548    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf"} err="failed to get container status \"2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf\": rpc error: code = NotFound desc = could not find container \"2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf\": container with ID starting with 2d82223e986cc6d77f7eb36018b7a5567dc9514a92db61adfb10cc1eac6526cf not found: ID does not exist"
Jul 27 20:23:11.435191 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.435158    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"]
Jul 27 20:23:11.439115 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:11.439094    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e837b8-c96f9696-x9zzw"]
Jul 27 20:23:12.488982 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:12.488948    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26539b65-f182-444b-b41f-ccd923c9b815" path="/var/lib/kubelet/pods/26539b65-f182-444b-b41f-ccd923c9b815/volumes"
Jul 27 20:23:15.485130 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:15.485092    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"]
Jul 27 20:23:15.487288 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:15.487232    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e837b8-lvdpj"]
Jul 27 20:23:15.487937 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:15.487903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:15.877505 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:15.877397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:16.489265 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:16.489224    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c6823ec-5269-4b12-885d-8654ac2b9be0" path="/var/lib/kubelet/pods/6c6823ec-5269-4b12-885d-8654ac2b9be0/volumes"
Jul 27 20:23:21.751777 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:21.751738    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:21.852078 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:21.852043    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:22.039736 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:22.039645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:22.315190 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:22.315099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:28.389599 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:28.389566    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d6vdb_3f7a5d90-0445-447b-b992-2628efd922f9/manager/0.log"
Jul 27 20:23:28.512417 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:28.512390    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5744b65d6-pfq5n_2f49e78b-59ed-490e-bfe7-535443db2fbf/manager/0.log"
Jul 27 20:23:28.892768 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:28.892738    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787b9f94c8-6xt87_0fffa1c2-3491-46af-8e71-3ba88f3ca103/copy-manifests/0.log"
Jul 27 20:23:28.898626 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:28.898604    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787b9f94c8-6xt87_0fffa1c2-3491-46af-8e71-3ba88f3ca103/manager/0.log"
Jul 27 20:23:29.605056 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:29.605026    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753025-f5gct_282d2290-ab83-495b-973f-6092173a7aa8/cleanup/0.log"
Jul 27 20:23:29.723679 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:29.723651    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753040-722gm_390e5a62-bbea-444b-8f35-d818ef99b754/cleanup/0.log"
Jul 27 20:23:29.940039 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:29.940011    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-dvbks_46db5e5c-3a3c-44c5-b2c9-bd7b484b2eff/postgres/0.log"
Jul 27 20:23:32.603801 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:32.603767    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-766b79c75b-jt2nv_029948f5-1e42-453e-8016-5e11292fe3c7/kube-auth-proxy/0.log"
Jul 27 20:23:32.981418 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:32.981389    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-82txl_26d2d9cd-b794-4ae7-97e9-d5bb72da28af/payload-pre-processing/0.log"
Jul 27 20:23:33.581961 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:33.581934    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2hgqk_3bd7c5c8-286b-4ef8-b54e-99a212687321/main/0.log"
Jul 27 20:23:33.812827 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:33.812800    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-spj8b_05394dcf-a888-42bd-9ba7-04440a5e6c09/main/0.log"
Jul 27 20:23:34.161943 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:34.161906    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-mqcz5_07b7d2d7-d7ba-44c9-95c5-03b8e5ed9800/main/0.log"
Jul 27 20:23:39.944195 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.944161    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8ngh2/must-gather-pdl5x"]
Jul 27 20:23:39.944599 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.944575    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c6823ec-5269-4b12-885d-8654ac2b9be0" containerName="revoke-keys"
Jul 27 20:23:39.944599 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.944589    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c6823ec-5269-4b12-885d-8654ac2b9be0" containerName="revoke-keys"
Jul 27 20:23:39.944683 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.944618    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26539b65-f182-444b-b41f-ccd923c9b815" containerName="payload-pre-processing"
Jul 27 20:23:39.944683 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.944623    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="26539b65-f182-444b-b41f-ccd923c9b815" containerName="payload-pre-processing"
Jul 27 20:23:39.944683 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.944680    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="26539b65-f182-444b-b41f-ccd923c9b815" containerName="payload-pre-processing"
Jul 27 20:23:39.944780 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.944691    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c6823ec-5269-4b12-885d-8654ac2b9be0" containerName="revoke-keys"
Jul 27 20:23:39.948059 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.948039    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ngh2/must-gather-pdl5x"
Jul 27 20:23:39.950344 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.950322    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8ngh2\"/\"kube-root-ca.crt\""
Jul 27 20:23:39.950447 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.950397    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8ngh2\"/\"openshift-service-ca.crt\""
Jul 27 20:23:39.955403 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.955315    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ngh2/must-gather-pdl5x"]
Jul 27 20:23:39.960941 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.960918    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:39.999661 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.999630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8jcx\" (UniqueName: \"kubernetes.io/projected/cfd2c2f4-5b52-4974-bb37-08a14153c273-kube-api-access-x8jcx\") pod \"must-gather-pdl5x\" (UID: \"cfd2c2f4-5b52-4974-bb37-08a14153c273\") " pod="openshift-must-gather-8ngh2/must-gather-pdl5x"
Jul 27 20:23:39.999837 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:39.999805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfd2c2f4-5b52-4974-bb37-08a14153c273-must-gather-output\") pod \"must-gather-pdl5x\" (UID: \"cfd2c2f4-5b52-4974-bb37-08a14153c273\") " pod="openshift-must-gather-8ngh2/must-gather-pdl5x"
Jul 27 20:23:40.061770 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.061734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log" limit=1024
Jul 27 20:23:40.100523 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.100486    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfd2c2f4-5b52-4974-bb37-08a14153c273-must-gather-output\") pod \"must-gather-pdl5x\" (UID: \"cfd2c2f4-5b52-4974-bb37-08a14153c273\") " pod="openshift-must-gather-8ngh2/must-gather-pdl5x"
Jul 27 20:23:40.100714 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.100532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8jcx\" (UniqueName: \"kubernetes.io/projected/cfd2c2f4-5b52-4974-bb37-08a14153c273-kube-api-access-x8jcx\") pod \"must-gather-pdl5x\" (UID: \"cfd2c2f4-5b52-4974-bb37-08a14153c273\") " pod="openshift-must-gather-8ngh2/must-gather-pdl5x"
Jul 27 20:23:40.100836 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.100814    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfd2c2f4-5b52-4974-bb37-08a14153c273-must-gather-output\") pod \"must-gather-pdl5x\" (UID: \"cfd2c2f4-5b52-4974-bb37-08a14153c273\") " pod="openshift-must-gather-8ngh2/must-gather-pdl5x"
Jul 27 20:23:40.108236 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.108213    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8jcx\" (UniqueName: \"kubernetes.io/projected/cfd2c2f4-5b52-4974-bb37-08a14153c273-kube-api-access-x8jcx\") pod \"must-gather-pdl5x\" (UID: \"cfd2c2f4-5b52-4974-bb37-08a14153c273\") " pod="openshift-must-gather-8ngh2/must-gather-pdl5x"
Jul 27 20:23:40.258481 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.258400    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ngh2/must-gather-pdl5x"
Jul 27 20:23:40.388743 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.388712    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ngh2/must-gather-pdl5x"]
Jul 27 20:23:40.391384 ip-10-0-131-59 kubenswrapper[2579]: W0727 20:23:40.391352    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfd2c2f4_5b52_4974_bb37_08a14153c273.slice/crio-aee80c06d7f44994de9f2a1569f312e3a9db77f86cca3c50297e7f672e0c28aa WatchSource:0}: Error finding container aee80c06d7f44994de9f2a1569f312e3a9db77f86cca3c50297e7f672e0c28aa: Status 404 returned error can't find the container with id aee80c06d7f44994de9f2a1569f312e3a9db77f86cca3c50297e7f672e0c28aa
Jul 27 20:23:40.393089 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.393072    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 20:23:40.567679 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:40.567594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ngh2/must-gather-pdl5x" event={"ID":"cfd2c2f4-5b52-4974-bb37-08a14153c273","Type":"ContainerStarted","Data":"aee80c06d7f44994de9f2a1569f312e3a9db77f86cca3c50297e7f672e0c28aa"}
Jul 27 20:23:41.576625 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:41.576541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ngh2/must-gather-pdl5x" event={"ID":"cfd2c2f4-5b52-4974-bb37-08a14153c273","Type":"ContainerStarted","Data":"d40959ea36843715b755f371f52822c0a745467c198dbef512f3e9100ffcbfd8"}
Jul 27 20:23:41.576625 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:41.576588    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ngh2/must-gather-pdl5x" event={"ID":"cfd2c2f4-5b52-4974-bb37-08a14153c273","Type":"ContainerStarted","Data":"a00606f7b0fc3be6bb236675dc456ae98b27a9fb6efd897fbd92c36bb9ddc78b"}
Jul 27 20:23:41.591846 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:41.591778    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8ngh2/must-gather-pdl5x" podStartSLOduration=1.664397107 podStartE2EDuration="2.591759426s" totalImagesPullingTime="927.362319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:23:41.591522209 +0000 UTC m=+3473.682873708" watchObservedRunningTime="2026-07-27 20:23:41.591759426 +0000 UTC m=+3473.683110926"
Jul 27 20:23:42.880328 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:42.880302    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wtrmw_185be14c-e7d0-494b-b626-94f381a9ab93/global-pull-secret-syncer/0.log"
Jul 27 20:23:42.973638 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:42.973605    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dx8ff_b3fc4e27-9510-4feb-a0db-a8c1b320db57/konnectivity-agent/0.log"
Jul 27 20:23:43.097732 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:43.097699    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-59.ec2.internal_94b0f96561dcf348510b71c7e517149a/haproxy/0.log"
Jul 27 20:23:49.000612 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:49.000468    2579 scope.go:117] "RemoveContainer" containerID="86fdd2acb5bc48696ca604eace8d2e086f051ba51c9941697ff7321a114e62ad"
Jul 27 20:23:50.650712 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.650623    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7930482-e1d2-42f5-b2e9-bb5966c50ab7/alertmanager/0.log"
Jul 27 20:23:50.674853 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.674824    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7930482-e1d2-42f5-b2e9-bb5966c50ab7/config-reloader/0.log"
Jul 27 20:23:50.696633 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.696604    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7930482-e1d2-42f5-b2e9-bb5966c50ab7/kube-rbac-proxy-web/0.log"
Jul 27 20:23:50.721384 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.721357    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7930482-e1d2-42f5-b2e9-bb5966c50ab7/kube-rbac-proxy/0.log"
Jul 27 20:23:50.745121 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.745032    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7930482-e1d2-42f5-b2e9-bb5966c50ab7/kube-rbac-proxy-metric/0.log"
Jul 27 20:23:50.768675 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.768636    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7930482-e1d2-42f5-b2e9-bb5966c50ab7/prom-label-proxy/0.log"
Jul 27 20:23:50.799038 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.798872    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7930482-e1d2-42f5-b2e9-bb5966c50ab7/init-config-reloader/0.log"
Jul 27 20:23:50.892088 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.892048    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-jrckg_95b0add7-044b-44ca-a617-164947d0f3e7/kube-state-metrics/0.log"
Jul 27 20:23:50.911507 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.911419    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-jrckg_95b0add7-044b-44ca-a617-164947d0f3e7/kube-rbac-proxy-main/0.log"
Jul 27 20:23:50.933806 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.933784    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-jrckg_95b0add7-044b-44ca-a617-164947d0f3e7/kube-rbac-proxy-self/0.log"
Jul 27 20:23:50.959633 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:50.959595    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-854b86fcd8-mxfbr_d5780864-eb8b-462d-9bcc-0b53d408bd52/metrics-server/0.log"
Jul 27 20:23:51.187971 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.187942    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rknpk_1e20c8ae-3746-42be-ae69-c9077883b641/node-exporter/0.log"
Jul 27 20:23:51.215361 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.215337    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rknpk_1e20c8ae-3746-42be-ae69-c9077883b641/kube-rbac-proxy/0.log"
Jul 27 20:23:51.236360 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.236340    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rknpk_1e20c8ae-3746-42be-ae69-c9077883b641/init-textfile/0.log"
Jul 27 20:23:51.267419 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.267390    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6pnf4_fc5733ad-861c-4143-abd6-e88310264296/kube-rbac-proxy-main/0.log"
Jul 27 20:23:51.311762 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.311716    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6pnf4_fc5733ad-861c-4143-abd6-e88310264296/kube-rbac-proxy-self/0.log"
Jul 27 20:23:51.334127 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.334042    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6pnf4_fc5733ad-861c-4143-abd6-e88310264296/openshift-state-metrics/0.log"
Jul 27 20:23:51.374391 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.374365    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c01d3c88-06e2-4885-a3c6-06cb52f931df/prometheus/0.log"
Jul 27 20:23:51.396828 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.396798    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c01d3c88-06e2-4885-a3c6-06cb52f931df/config-reloader/0.log"
Jul 27 20:23:51.419743 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.419719    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c01d3c88-06e2-4885-a3c6-06cb52f931df/thanos-sidecar/0.log"
Jul 27 20:23:51.440967 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.440880    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c01d3c88-06e2-4885-a3c6-06cb52f931df/kube-rbac-proxy-web/0.log"
Jul 27 20:23:51.462874 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.462846    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c01d3c88-06e2-4885-a3c6-06cb52f931df/kube-rbac-proxy/0.log"
Jul 27 20:23:51.489487 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.489456    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c01d3c88-06e2-4885-a3c6-06cb52f931df/kube-rbac-proxy-thanos/0.log"
Jul 27 20:23:51.511333 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.511303    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c01d3c88-06e2-4885-a3c6-06cb52f931df/init-config-reloader/0.log"
Jul 27 20:23:51.541416 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.541388    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-qgb82_dbaaea17-1a2c-406a-a748-03b901948ead/prometheus-operator/0.log"
Jul 27 20:23:51.561100 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.561071    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-qgb82_dbaaea17-1a2c-406a-a748-03b901948ead/kube-rbac-proxy/0.log"
Jul 27 20:23:51.589820 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.589783    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-8rf2s_ba4786db-13fb-41ce-bfe3-d564d2ab98c9/prometheus-operator-admission-webhook/0.log"
Jul 27 20:23:51.699467 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.699393    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc687cfd-5xxbj_fa44b356-c08d-49ab-997c-f61420b24564/thanos-query/0.log"
Jul 27 20:23:51.720030 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.719959    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc687cfd-5xxbj_fa44b356-c08d-49ab-997c-f61420b24564/kube-rbac-proxy-web/0.log"
Jul 27 20:23:51.740987 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.740956    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc687cfd-5xxbj_fa44b356-c08d-49ab-997c-f61420b24564/kube-rbac-proxy/0.log"
Jul 27 20:23:51.762461 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.762435    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc687cfd-5xxbj_fa44b356-c08d-49ab-997c-f61420b24564/prom-label-proxy/0.log"
Jul 27 20:23:51.791547 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.791520    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc687cfd-5xxbj_fa44b356-c08d-49ab-997c-f61420b24564/kube-rbac-proxy-rules/0.log"
Jul 27 20:23:51.812379 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:51.812350    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc687cfd-5xxbj_fa44b356-c08d-49ab-997c-f61420b24564/kube-rbac-proxy-metrics/0.log"
Jul 27 20:23:53.155833 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.155795    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"]
Jul 27 20:23:53.164720 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.164687    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.166456 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.166119    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"]
Jul 27 20:23:53.167118 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.167091    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8ngh2\"/\"default-dockercfg-gkqfh\""
Jul 27 20:23:53.254589 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.254533    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-podres\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.254790 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.254606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-lib-modules\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.254790 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.254669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-proc\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.254928 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.254833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6kk5\" (UniqueName: \"kubernetes.io/projected/d46ba987-2c00-4f2f-93e0-a69552b8d02c-kube-api-access-k6kk5\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.254928 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.254909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-sys\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356314 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-sys\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356493 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-podres\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356493 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-lib-modules\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356493 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356409    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-sys\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356493 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-proc\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356493 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356474    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-proc\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356828 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356601    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-lib-modules\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356828 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6kk5\" (UniqueName: \"kubernetes.io/projected/d46ba987-2c00-4f2f-93e0-a69552b8d02c-kube-api-access-k6kk5\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.356828 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.356681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d46ba987-2c00-4f2f-93e0-a69552b8d02c-podres\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.365495 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.365472    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6kk5\" (UniqueName: \"kubernetes.io/projected/d46ba987-2c00-4f2f-93e0-a69552b8d02c-kube-api-access-k6kk5\") pod \"perf-node-gather-daemonset-s7ns8\" (UID: \"d46ba987-2c00-4f2f-93e0-a69552b8d02c\") " pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.482463 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.482425    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:53.692820 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:53.692493    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"]
Jul 27 20:23:54.684804 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:54.684758    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8" event={"ID":"d46ba987-2c00-4f2f-93e0-a69552b8d02c","Type":"ContainerStarted","Data":"cdf119b0645921261f0065f5e556390ad3fd489827983522bc298772ca482c05"}
Jul 27 20:23:54.685309 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:54.684846    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:23:54.685309 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:54.684866    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8" event={"ID":"d46ba987-2c00-4f2f-93e0-a69552b8d02c","Type":"ContainerStarted","Data":"fcb15c6ac545e0d133d886ae809c58be055ae840058888ec9cf66f67899b278b"}
Jul 27 20:23:54.702010 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:54.701942    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8" podStartSLOduration=1.701920817 podStartE2EDuration="1.701920817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:23:54.69944708 +0000 UTC m=+3486.790798580" watchObservedRunningTime="2026-07-27 20:23:54.701920817 +0000 UTC m=+3486.793272317"
Jul 27 20:23:55.888386 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:55.888354    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qd2hp_3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a/dns/0.log"
Jul 27 20:23:55.911112 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:55.911086    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qd2hp_3737a0cf-1d14-4e8e-9f28-dc12fd2ff68a/kube-rbac-proxy/0.log"
Jul 27 20:23:55.956841 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:55.956808    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vzl59_3ab06202-6bd0-4bd7-be2e-c4a4ac85a1e9/dns-node-resolver/0.log"
Jul 27 20:23:56.625713 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:56.625683    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-spvgm_9138d8cc-fdd4-4883-84be-0600b09f69a8/node-ca/0.log"
Jul 27 20:23:57.750224 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:57.750196    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-766b79c75b-jt2nv_029948f5-1e42-453e-8016-5e11292fe3c7/kube-auth-proxy/0.log"
Jul 27 20:23:57.889127 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:57.889097    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-82txl_26d2d9cd-b794-4ae7-97e9-d5bb72da28af/payload-pre-processing/0.log"
Jul 27 20:23:58.768469 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:58.768441    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s2rbr_c8c1a9a5-7b01-4855-a9f8-809dbf3c9e23/serve-healthcheck-canary/0.log"
Jul 27 20:23:59.428335 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:59.428303    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r872m_4d7f6643-4911-4092-9a15-38019dc60868/kube-rbac-proxy/0.log"
Jul 27 20:23:59.452131 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:59.452104    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r872m_4d7f6643-4911-4092-9a15-38019dc60868/exporter/0.log"
Jul 27 20:23:59.474192 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:23:59.474165    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r872m_4d7f6643-4911-4092-9a15-38019dc60868/extractor/0.log"
Jul 27 20:24:00.703759 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:00.703727    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8ngh2/perf-node-gather-daemonset-s7ns8"
Jul 27 20:24:01.893845 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:01.892704    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d6vdb_3f7a5d90-0445-447b-b992-2628efd922f9/manager/0.log"
Jul 27 20:24:02.073769 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:02.073633    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5744b65d6-pfq5n_2f49e78b-59ed-490e-bfe7-535443db2fbf/manager/0.log"
Jul 27 20:24:02.242369 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:02.242336    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787b9f94c8-6xt87_0fffa1c2-3491-46af-8e71-3ba88f3ca103/manager/0.log"
Jul 27 20:24:02.264515 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:02.264483    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787b9f94c8-6xt87_0fffa1c2-3491-46af-8e71-3ba88f3ca103/copy-manifests/0.log"
Jul 27 20:24:03.664190 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:03.664165    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-766b79c75b-jt2nv_029948f5-1e42-453e-8016-5e11292fe3c7/kube-auth-proxy/0.log"
Jul 27 20:24:03.775086 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:03.775038    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-82txl_26d2d9cd-b794-4ae7-97e9-d5bb72da28af/payload-pre-processing/0.log"
Jul 27 20:24:03.827228 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:03.827174    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8cbf9cd8b-t4dv2_12c0ea9a-c39e-4a76-91bc-74b7de3b43e9/manager/0.log"
Jul 27 20:24:03.881087 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:03.881052    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hkhh4_1b0ad86e-b6ca-488c-bbfb-6528f33245fc/openshift-lws-operator/0.log"
Jul 27 20:24:05.332961 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:05.332929    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-766b79c75b-jt2nv_029948f5-1e42-453e-8016-5e11292fe3c7/kube-auth-proxy/0.log"
Jul 27 20:24:05.457779 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:05.457750    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-82txl_26d2d9cd-b794-4ae7-97e9-d5bb72da28af/payload-pre-processing/0.log"
Jul 27 20:24:10.239017 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.238938    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/kube-multus-additional-cni-plugins/0.log"
Jul 27 20:24:10.262020 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.261992    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/egress-router-binary-copy/0.log"
Jul 27 20:24:10.283736 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.283712    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/cni-plugins/0.log"
Jul 27 20:24:10.305212 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.305186    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/bond-cni-plugin/0.log"
Jul 27 20:24:10.326202 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.326165    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/routeoverride-cni/0.log"
Jul 27 20:24:10.348196 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.348165    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/whereabouts-cni-bincopy/0.log"
Jul 27 20:24:10.369267 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.369215    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hcqwt_56643b2e-0ea2-498b-aed2-e185018e7028/whereabouts-cni/0.log"
Jul 27 20:24:10.596315 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.596212    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/1.log"
Jul 27 20:24:10.628652 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.628605    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2v4h_bcd1a649-a8e2-4657-ba0d-b19301043af8/kube-multus/2.log"
Jul 27 20:24:10.750591 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.750555    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qc4zf_1681bd39-34f5-4469-8ed5-fc75330bad04/network-metrics-daemon/0.log"
Jul 27 20:24:10.767587 ip-10-0-131-59 kubenswrapper[2579]: I0727 20:24:10.767557    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qc4zf_1681bd39-34f5-4469-8ed5-fc75330bad04/kube-rbac-proxy/0.log"